CN109325422A - Expression recognition method, device, terminal and computer readable storage medium - Google Patents
Expression recognition method, device, terminal and computer readable storage medium Download PDFInfo
- Publication number
- CN109325422A CN109325422A CN201810990197.6A CN201810990197A CN109325422A CN 109325422 A CN109325422 A CN 109325422A CN 201810990197 A CN201810990197 A CN 201810990197A CN 109325422 A CN109325422 A CN 109325422A
- Authority
- CN
- China
- Prior art keywords
- expression
- expressive features
- recognition
- time series
- expression recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The application provides a kind of expression recognition method, multiple facial images continuous in time including same face in determination video to be identified；The expressive features of the multiple facial images of average value of expression value are extracted respectively；Expressive features time series is determined according to the sequential relationship of the multiple facial images of the average value of expression value and each expressive features；The average value expressive features time series of expression value is input to preset Expression Recognition model, determine expression classification corresponding with the average value expressive features time series of expression value, and using the corresponding expression of average value expression classification of expression value as Expression Recognition result.The application method, since expressive features time series can react the timing feature of expression in the video, the Expression Recognition result identified based on the Expression Recognition model that expressive features time series is established is more acurrate.Present invention also provides a kind of expression recognition apparatus, terminal and computer readable storage mediums.
Description
Technical field
This application involves computer image processing technology fields, specifically, this application involves a kind of expression recognition method,
Device, terminal and computer readable storage medium.
Background technique
In the prior art, Expression Recognition is normally based on the static recognition methods of geometric error modeling feature, i.e. acquisition single width waits for
It identifies image, then extracts the expressive features in images to be recognized, and then match in preset expression library according to expressive features
To with the matched expression of expressive features, realize to the Expression Recognition in images to be recognized.
But since the expression in video has timing, i.e., the expression in video in each frame image has subtle difference
Not, therefore, expression in video is identified based on expression recognition method in the prior art, can only be identified in single image
Expression, and since single image can not react the timing feature of expression, thus can not be to the nuance of expression in video
It is accurately identified, and then identifies the expression result inaccuracy that expression obtains in video, and due to existing expression recognition method
Accurate identification cannot be made for the slight change in expression, lead to the table that may recognize that by existing expression recognition method
Affectionate person's class is few.
Therefore, technological deficiency in the prior art is: existing expression recognition method identification is in single static image
Expression, the expression in video can not be accurately identified, and the expression classification identified is few.
Summary of the invention
The purpose of the application is intended at least can solve above-mentioned one of technological deficiency, especially existing expression recognition method
What is identified is the expression in single static image, can not accurately identify the technological deficiency of expression in video.
In a first aspect, the application provides a kind of expression recognition method, this method comprises the following steps:
Determine multiple facial images continuous in time of same face in video to be identified；
The expressive features of multiple facial images are extracted respectively；
Expressive features time series is determined according to the sequential relationship of multiple facial images and each expressive features；
Expressive features time series is input to preset Expression Recognition model, determination is corresponding with expressive features time series
Expression classification, and using the corresponding expression of expression classification as Expression Recognition result.
Second aspect, the application also provide a kind of expression recognition apparatus, which includes:
Image determining module, for determining multiple face figures continuous in time of same face in video to be identified
Picture；
Characteristic extracting module, for extracting the expressive features of multiple facial images respectively；
Time series determining module determines table for the sequential relationship and each expressive features according to multiple facial images
Feelings feature time series；
Expression Recognition module, for expressive features time series to be input to preset Expression Recognition model, determining and table
The corresponding expression classification of feelings feature time series, and using the corresponding expression of expression classification as Expression Recognition result.
The third aspect, the application also provide a kind of Expression Recognition terminal, which includes: processor, memory and bus；
Bus, for connecting processor and memory；Memory, for storing operational order；Processor, for being referred to by call operation
It enables, executes the corresponding operation of the method as shown in the first aspect of the application.
Fourth aspect, this application provides a kind of computer readable storage medium, which is stored at least one
Instruction, at least one section of program, code set or instruction set, at least one instruction, at least one section of program, code set or instruction set by
Reason device is loaded and is executed in the method as shown in the first aspect of the application of realization.
The advantages of the application: multiple facial images continuous in time of same face in video to be identified are determined；Point
Indescribably take the expressive features of multiple facial images；Expression is determined according to the sequential relationship of multiple facial images and each expressive features
Feature time series；And then expressive features time series is input to preset Expression Recognition model, when determination is with expressive features
The corresponding expression classification of sequence sequence, and using the corresponding expression of expression classification as Expression Recognition result；In above scheme, it is based on table
Feelings feature time series establishes Expression Recognition model, is identified by the Expression Recognition model to the expression in video, by
The timing feature of expression in the video can be reflected in expressive features time series, therefore can based on expressive features time series
The slight change of expression in video is identified, so that Expression Recognition result is more acurrate, and due to the slight change of expression, so that can
The expression classification of identification is richer, therefore the Expression Recognition model based on the training of timing expression characteristic sequence may recognize that more multilist
The expression of feelings classification.
The additional aspect of the application and advantage will be set forth in part in the description, these will become from the following description
It obtains obviously, or recognized by the practice of the application.
Detailed description of the invention
The application is above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, in which:
Fig. 1 is a kind of method flow diagram of expression recognition method first embodiment provided by the present application；
Fig. 2 is a kind of method flow diagram of expression recognition method second embodiment provided by the present application；
Fig. 3 is a kind of structural schematic diagram of expression recognition apparatus provided by the present application；
Fig. 4 is a kind of structural schematic diagram of Expression Recognition terminal provided by the present application.
Specific embodiment
Embodiments herein is described below in detail, the example of embodiment is shown in the accompanying drawings, wherein identical from beginning to end
Or similar label indicates same or similar element or element with the same or similar functions.It is retouched below with reference to attached drawing
The embodiment stated is exemplary, and is only used for explaining the application, and cannot be construed to the limitation to the application.
Those skilled in the art of the present technique are appreciated that unless otherwise defined, all terms used herein (including technology art
Language and scientific term), there is meaning identical with the general understanding of those of ordinary skill in the application fields.Should also
Understand, those terms such as defined in the general dictionary, it should be understood that have in the context of the prior art
The consistent meaning of meaning, and unless idealization or meaning too formal otherwise will not be used by specific definitions as here
To explain.
Those skilled in the art of the present technique are appreciated that unless expressly stated, singular " one " used herein, "one"
It may also comprise plural form with "the".It is to be further understood that wording " comprising " used in the description of the present application is
Refer to existing characteristics, integer, step, operation, element and/or component, but it is not excluded that in the presence of or addition it is one or more other
Feature, integer, step, operation, element, component and/or their group.It should be understood that when we claim element to be " connected " or " coupling
Connect " to another element when, it can be directly connected or coupled to other elements, or there may also be intermediary elements.In addition, this
In " connection " or " coupling " that uses may include being wirelessly connected or wireless coupling.Wording "and/or" used herein includes one
A or more associated whole for listing item or any cell and all combination.
To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with attached drawing to the application embodiment party
Formula is described in further detail.
Expression recognition method, device, terminal and computer readable storage medium provided by the present application, it is intended to solve existing skill
The technical problem as above of art.
How the technical solution of the application and the technical solution of the application are solved with specifically embodiment below above-mentioned
Technical problem is described in detail.These specific embodiments can be combined with each other below, for the same or similar concept
Or process may repeat no more in certain embodiments.Below in conjunction with attached drawing, embodiments herein is described.
Embodiment one
This application provides a kind of expression recognition methods, as shown in Figure 1, this method comprises:
Step S101 determines multiple facial images continuous in time of same face in video to be identified.
Wherein, video to be identified can for image capture device acquisition video, and image capture device may be provided at it is each
Under application scenarios, then multiple facial images of the Time Continuous under each application scenarios can be obtained by image capture device.
Step S102 extracts the expressive features of multiple facial images respectively.
Wherein, the expressive features in facial image are extracted with existing feature extracting method.
Step S103 determines expressive features timing sequence according to the sequential relationship of multiple facial images and each expressive features
Column.
Wherein, expression temporal aspect sequence can react the sequential relationship of the corresponding expressive features of each facial image.
Expressive features time series is input to preset Expression Recognition model, when determination is with expressive features by step S104
The corresponding expression classification of sequence sequence, and using the corresponding expression of expression classification as Expression Recognition result.
Wherein, preset Expression Recognition model is established based on expressive features time series, for determining expressive features
The corresponding expression classification of time series.
The scheme recorded in the present embodiment as a result, determines the multiple continuous in time of same face in video to be identified
Facial image；The expressive features of multiple facial images are extracted respectively；According to the sequential relationship of multiple facial images and each table
Feelings feature determines expressive features time series；And then expressive features time series is input to preset Expression Recognition model, really
Fixed expression classification corresponding with expressive features time series, and using the corresponding expression of expression classification as Expression Recognition result；On
It states in scheme, Expression Recognition model is established based on expressive features time series, by the Expression Recognition model come in video
Expression is identified, since expressive features time series can reflect the timing feature of expression in the video, is based on table
Feelings feature time series may recognize that the slight change of expression in video, so that Expression Recognition result is more acurrate, and due to expression
Slight change so that identifiable expression classification is richer, therefore the Expression Recognition based on the training of timing expression characteristic sequence
Model may recognize that the expression of more multiple expression classification.
Embodiment two
The embodiment of the present application provides alternatively possible implementation, further includes implementing on the basis of example 1
Method shown in example two:
Further, step S101 determines multiple face figures continuous in time of same face in video to be identified
As after, further includes:
Multiple facial images are handled, multiple facial images that obtain that treated.
Wherein, carrying out processing to multiple facial images includes image noise reduction, the processing such as cuts, so that treated multiple people
Face image is apparent, and format is more unified, is convenient for subsequent processing.
Further, in step S103, expression is determined according to the sequential relationship of multiple facial images and each expressive features
Feature time series, comprising:
According to the sequential relationship of multiple facial images, the corresponding expressive features of each facial image are combined into expression timing
Characteristic sequence.
Wherein, since the corresponding expressive features of multiple facial images have timing, the expressive features that will be extracted
According to sequential combination at expressive features time series, with the timing feature of expressive features time series reaction expression.
Further, as shown in Fig. 2, the training method of preset Expression Recognition model, specifically includes:
Step S201 determines multiple facial images of the corresponding Time Continuous of expression video in training set.
Step S202 extracts the expressive features in each facial image.
Step S203, by the expressive features of each facial image according to sequential combination at expressive features time series.
Step S204, expressive features time series and expression classification to mark in advance are input to and follow as training sample
Ring neural network model carries out feature training to Recognition with Recurrent Neural Network model, until convergence, obtains Expression Recognition model.
Wherein, since the expression in different expression classifications corresponds to different expressive features, when different expressive features
Sequence sequence corresponds to different expressions, and expressive features time series is input to Recognition with Recurrent Neural Network model and carries out feature training, instruction
Practice and complete to obtain Expression Recognition model, Expression Recognition model is enabled effectively to judge the corresponding table of expressive features time series
Feelings classification.
Further, before carrying out feature training to Recognition with Recurrent Neural Network model, the multistage for being used for training pattern is collected
Expression video is right according to the corresponding expression classification of different expressive features time series using the multistage expression video as training set
Each expression classification carries out classification marker in the training set, obtains training sample, while marking the expectation point of the training sample
Class is as a result, the i.e. corresponding desired output expression classification of expressive features time series.Then, the training sample input after label is followed
In ring neural network model, Recognition with Recurrent Neural Network model is trained, until convergence, obtains Expression Recognition model.
Further, feature training is carried out to Recognition with Recurrent Neural Network model, until convergence, obtains Expression Recognition model,
It specifically includes:
Determine that training sample belongs to the probability value of each expression classification.
Using the corresponding expression of expression classification of maximum probability value as output result.
The expectation point that output result and training sample are demarcated in advance by the loss function in Recognition with Recurrent Neural Network model
Class result is compared, and when the output result of loss function is less than predetermined threshold, the training of training sample terminates.
Otherwise, training sample is re-entered into Recognition with Recurrent Neural Network model and is trained, led to before re -training
Inverse algorithms are crossed, each weight of Recognition with Recurrent Neural Network model is adjusted.Wherein, according to extraction expressive features time series
Cos distance calculate the probability value for belonging to each expression classification of the training sample, and the training sample is classified to probability value
In highest classification results, the corresponding expression of the classification results is the output result of training sample.Pass through Recognition with Recurrent Neural Network
Loss function in model is compared desired classification results with output result, makes a reservation for when the output result of loss function is less than
When threshold value, the training of the training sample terminates；Otherwise, which is re-entered into Recognition with Recurrent Neural Network model and is carried out
Training is adjusted each weight of Recognition with Recurrent Neural Network model, repeatedly by inverse algorithms before re -training
Until the output result of loss function is less than predetermined threshold, the training of the training sample terminates；Through the above way by training set
In thousands of training sample persistently to the Recognition with Recurrent Neural Network model carry out repetition training, until the model output expectation
The success rate of classification results reaches that people want as a result, at this point, the Recognition with Recurrent Neural Network model training obtains expression to restraining
Identification model can accurately carry out the judgement that expressive features time series corresponds to expression classification.
The scheme recorded in the present embodiment as a result, is based on Recognition with Recurrent Neural Network model foundation Expression Recognition model, by expression
Input of the feature time series as model passes through training using the corresponding expression of corresponding expression classification as the output of model
Obtained Expression Recognition model can be to multiple facial images progress Expression Recognition of Time Continuous in video, due to the Time Continuous
Multiple facial images expression timing so that the expression result identified is more acurrate；And based on a large amount of training samples to this
Recognition with Recurrent Neural Network model carries out repetition training, improves the precision of the model, further such that passing through the Expression Recognition model
Identify that the obtained corresponding Expression Recognition result of expression classification is more acurrate.
It further, include the expression video of tens of kinds of expression classifications in training sample, so that the expression trained is known
Other model may recognize that the expression of more multiple expression classification, wherein the expression video of tens of kinds of expression classifications in the training sample
It is to be determined based on the theories of psychology, every section of expression video corresponds to a kind of expression of expression classification, is marked by the profession after trained
Personnel are labeled different expression classifications according to expressive features time series, wherein expressive features time series be with
The feature of timing combines.
Wherein, 54 kinds of expressions include: it is happy, optimistic, admire, feel grateful, love, sincerity, vigor, trust, serenity, harmony,
Tolerance, thirst for, pride, it is conceited, calm, angry, brave, false, disdain, disobey, seeking to do others down, hating, detesting, insincerity, worry,
Envy, suspect, do not agree with, pessimism, conflict, humiliate, is sad, ignore, is tired, dejected, divert attention, is nervous, is worried, surrendering,
Regret deeply, be ashamed, grievance, it is boring, nervous, passive, frightened, revere, puzzle, embarrassment, cowardice, surprised, interest, expectation, face is without table
Feelings.
Wherein, which can be the different expressive features according to 54 kinds of expressions, by crawler technology in webpage
Swash the human face expression video got.Alternatively, to be shot simultaneously by 54 kinds of different micro- expressions to experimenter's face
Mark obtained video.The Expression Recognition model obtained as a result, based on above-mentioned 54 kinds of expressions progress model training can be to a variety of micro-
Expression is identified.
Further, the expression video in training set is stored in the form of two-dimentional table structure with corresponding expression classification.
Wherein, in training set include the corresponding expression video of a variety of expression classifications, by expression video and corresponding expression
Classification is stored in the form of two-dimentional table structure, i.e., expression video is expressed as a two dimension with the data structure of corresponding expression classification
Table, convenient for the management to expression video data and corresponding expression classification.
Further, this method further include:
The expressive features of multiple facial images face characteristic corresponding with known identities information is compared, determines expressive features
Identity information whether be known identities information.
By the expressive features in the facial image that extracts, face characteristic corresponding with known identities information compares,
If it does, then can determine that the corresponding identity information of the expressive features is consistent with known identities information, otherwise, it is determined that the expression is special
It levies corresponding identity information and known identities information is inconsistent, that is, can determine whether the corresponding people of the expressive features and known identities information
Whether corresponding people is the same person, such as under the application scenarios that ticket checking is entered the station, can based on the above method extract it is current this
Personal expressive features, are compared with the corresponding face characteristic of known identities information obtained on identity-based certificate, verify
Whether this current people is the corresponding people of known identities information on identity document.
Further, the expressive features of multiple facial images face characteristic corresponding with known identities information is compared,
Whether the identity information for determining expressive features is known identities information, comprising:
Calculate the similarity of the expressive features face characteristic corresponding with known identities information of multiple facial images.
Wherein, the method for calculating the similarity between characteristic point can be real by similarity calculating method in the prior art
The distance between it is existing, for example calculate two characteristic points.
When similarity be greater than preset threshold, determine the identity information of expressive features for known identities information, otherwise, it is determined that table
The identity information of feelings feature is not known identities information.
Further, this method further include:
According to Expression Recognition as a result, Expression Recognition result is matched with the expression in default fraud expression library,
Determine whether Expression Recognition result is fraud expression.
Wherein, the expression of a large amount of existing frauds of storage in fraud expression library is preset, wherein wrapping in fraud expression
It includes based on lift eyebrow, a variety of fraud expressions stared, narrow the facial expressions such as eye, corners of the mouth progress data evaluation of markers, such as: blink eyebrow frequency
The expression that rate is greater than preset threshold may indicate that the expression belongs to fraud expression.
Wherein, the fraud expression of every kind of expression classification further includes a variety of expression subclass, for example, lift eyebrow can be divided into interior angle,
Exterior angle；Lip can be divided into lip upwards, the corners of the mouth raises up, the corners of the mouth is tightened, the corners of the mouth sinks, the corners of the mouth stretches, lip is tightened, radian of opening one's mouth
Small, big 8 dimensions of radian of opening one's mouth；Therefore, based on the identification to expression in video, the Expression Recognition result identified is reflected
The slight change of expression, and then can be recognized accurately whether the expression is fraud expression.
Further, if Expression Recognition result is fraud expression, this method further include:
Prompting message corresponding with fraud expression is generated, and prompting message is sent to the terminal device of related personnel.
Wherein, by the expression being recognized accurately in video in face, it can further accurately identify whether the expression takes advantage of
Expression is cheated, if it is, can judge in time, and can effectively prevent the generation of fraud.
For example: for the client in financial industry, by the above method, if by the Expression Recognition result of the client
It is compared with the fraud expression in database, it is found that the client once has fraud, then show that the customers' credit is bad, can
The client is paid close attention to again, to determine whether the client has risk of fraud；Or prompting message will be sentenced and feed back to relevant people
The terminal device of member, to reinforce prevention awareness.
The scheme recorded in the present embodiment as a result, the expression identified based on Expression Recognition model is as a result, can further sentence
Whether the expression of breaking is fraud expression, so that the program can be widely used under different application scenarios, if it is decided that certain
The expression of people is fraud expression, and further processing can be made to this person, avoids making other frauds.
Embodiment three
The embodiment of the present application provides a kind of expression recognition apparatus 30, as shown in figure 3, the expression recognition apparatus 30 can wrap
It includes: image determining module 301, characteristic extracting module 302, time series determining module 303 and Expression Recognition module 304,
In,
Image determining module 301, for determining multiple faces continuous in time of same face in video to be identified
Image.
Characteristic extracting module 302, for extracting the expressive features of multiple facial images respectively.
Time series determining module 303 is determined for the sequential relationship and each expressive features according to multiple facial images
Expressive features time series.
Expression Recognition module 304, for expressive features time series to be input to preset Expression Recognition model, determine with
The corresponding expression classification of expressive features time series, and using the corresponding expression of expression classification as Expression Recognition result.
The scheme recorded in the present embodiment as a result, determines the multiple continuous in time of same face in video to be identified
Facial image；The expressive features of multiple facial images are extracted respectively；According to the sequential relationship of multiple facial images and each table
Feelings feature determines expressive features time series；And then expressive features time series is input to preset Expression Recognition model, really
Fixed expression classification corresponding with expressive features time series, and using the corresponding expression of expression classification as Expression Recognition result；On
It states in scheme, Expression Recognition model is established based on expressive features time series, by the Expression Recognition model come in video
Expression is identified, since expressive features time series can reflect the timing feature of expression in the video, is based on table
Feelings feature time series may recognize that the slight change of expression in video, so that Expression Recognition result is more acurrate, and due to expression
Slight change so that identifiable expression classification is richer, therefore the Expression Recognition based on the training of timing expression characteristic sequence
Model may recognize that the expression of more multiple expression classification.
Example IV
The embodiment of the present application provides alternatively possible implementation, further includes implementing on the basis of embodiment three
Scheme shown in example four, wherein
It further, further include image processing module 300 after image determining module 301, for multiple facial images
It is handled, multiple facial images that obtain that treated.
Wherein, carrying out processing to multiple facial images includes image noise reduction, the processing such as cuts, so that treated multiple people
Face image is apparent, and format is more unified, is convenient for subsequent processing.
Further, special according to the sequential relationship of multiple facial images and each expression in time series determining module 303
It levies and determines expressive features time series, comprising:
Expressive features time series is determined according to the sequential relationship of multiple facial images and each expressive features.
Wherein, since the corresponding expressive features of multiple facial images have timing, the expressive features that will be extracted
According to sequential combination at expressive features time series, with the timing feature of expressive features time series reaction expression.
It further, further include model training module 305, for training Expression Recognition model, model training module 305 is wrapped
Include image processing unit 3051, human facial feature extraction unit 3052, expressive features time series generation unit 3053 and model instruction
Practice unit 3054, wherein
Image determination unit 3051, for determining multiple face figures of the corresponding Time Continuous of expression video in training set
Picture.
Human facial feature extraction unit 3052, for extracting the expressive features in each facial image.
Expressive features time series generation unit 3053, for by the expressive features of each facial image according to sequential combination
At expressive features time series.
Model training unit 3054, for mark in advance expressive features time series and expression classification as training sample
This, is input to Recognition with Recurrent Neural Network model, carries out feature training to Recognition with Recurrent Neural Network model, until convergence, obtains expression knowledge
Other model.
Wherein, since the expression in different expression classifications corresponds to different expressive features, when different expressive features
Sequence sequence corresponds to different expressions, and expressive features time series is input to Recognition with Recurrent Neural Network model and carries out feature training, instruction
Practice and complete to obtain Expression Recognition model, Expression Recognition model is enabled effectively to judge the corresponding table of expressive features time series
Feelings classification.
Further, before carrying out feature training to Recognition with Recurrent Neural Network model, the multistage for being used for training pattern is collected
Expression video is right according to the corresponding expression classification of different expressive features time series using the multistage expression video as training set
Each expression classification carries out classification marker in the training set, obtains training sample, while marking the expectation point of the training sample
Class is as a result, the i.e. corresponding desired output expression classification of expressive features time series.Then, the training sample input after label is followed
In ring neural network model, Recognition with Recurrent Neural Network model is trained, until convergence, obtains Expression Recognition model.
Determine that training sample belongs to the probability value of each expression classification.
Using the corresponding expression of expression classification of maximum probability value as output result.
The expectation point that output result and training sample are demarcated in advance by the loss function in Recognition with Recurrent Neural Network model
Class result is compared, and when the output result of loss function is less than predetermined threshold, the training of training sample terminates.
Otherwise, training sample is re-entered into Recognition with Recurrent Neural Network model and is trained, led to before re -training
Inverse algorithms are crossed, each weight of Recognition with Recurrent Neural Network model is adjusted.Wherein, according to extraction expressive features time series
Cos distance calculate the probability value for belonging to each expression classification of the training sample, and the training sample is classified to probability value
In highest classification results, the corresponding expression of the classification results is the output result of training sample.Pass through Recognition with Recurrent Neural Network
Loss function in model is compared desired classification results with output result, makes a reservation for when the output result of loss function is less than
When threshold value, the training of the training sample terminates；Otherwise, which is re-entered into Recognition with Recurrent Neural Network model and is carried out
Training is adjusted each weight of Recognition with Recurrent Neural Network model, repeatedly by inverse algorithms before re -training
Until the output result of loss function is less than predetermined threshold, the training of the training sample terminates；Through the above way by training set
In thousands of training sample persistently to the Recognition with Recurrent Neural Network model carry out repetition training, until the model output expectation
The success rate of classification results reaches that people want as a result, at this point, the Recognition with Recurrent Neural Network model training obtains expression to restraining
Identification model can accurately carry out the judgement that expressive features time series corresponds to expression classification.
The scheme recorded in the present embodiment as a result, is based on Recognition with Recurrent Neural Network model foundation Expression Recognition model, by expression
Input of the feature time series as model passes through training using the corresponding expression of corresponding expression classification as the output of model
Obtained Expression Recognition model can be to multiple facial images progress Expression Recognition of Time Continuous in video, due to the Time Continuous
Multiple facial images expression timing so that the expression result identified is more acurrate；And based on a large amount of training samples to this
Recognition with Recurrent Neural Network model carries out repetition training, improves the precision of the model, further such that passing through the Expression Recognition model
Identify that the obtained corresponding Expression Recognition result of expression classification is more acurrate.
It further, include the expression video of tens of kinds of expression classifications in training sample, so that the expression trained is known
Other model may recognize that the expression of more multiple expression classification, wherein the expression video of tens of kinds of expression classifications in the training sample
It is to be determined based on the theories of psychology, every section of expression video corresponds to a kind of expression of expression classification, is marked by the profession after trained
Personnel are labeled different expression classifications according to expressive features time series, wherein expressive features time series be with
The feature of timing combines.
Wherein, 54 kinds of expressions include: it is happy, optimistic, admire, feel grateful, love, sincerity, vigor, trust, serenity, harmony,
Tolerance, thirst for, pride, it is conceited, calm, angry, brave, false, disdain, disobey, seeking to do others down, hating, detesting, insincerity, worry,
Envy, suspect, do not agree with, pessimism, conflict, humiliate, is sad, ignore, is tired, dejected, divert attention, is nervous, is worried, surrendering,
Regret deeply, be ashamed, grievance, it is boring, nervous, passive, frightened, revere, puzzle, embarrassment, cowardice, surprised, interest, expectation, face is without table
Feelings.
Wherein, which can be the different expressive features according to 54 kinds of expressions, by crawler technology in webpage
Swash the human face expression video got.Alternatively, to be shot simultaneously by 54 kinds of different micro- expressions to experimenter's face
Mark obtained video.The Expression Recognition model obtained as a result, based on above-mentioned 54 kinds of expressions progress model training can be to a variety of micro-
Expression is identified.
Further, the expression video in training set is stored in the form of two-dimentional table structure with corresponding expression.
Wherein, in training set include the corresponding expression video of a variety of expression classifications, by expression video and corresponding expression
Classification is stored in the form of two-dimentional table structure, i.e., expression video is expressed as a two dimension with the data structure of corresponding expression classification
Table, convenient for the management to expression video data and corresponding expression classification.
Further, which further includes authentication module 306, for by the expressive features of multiple facial images and
Know that identity information corresponds to face characteristic and compares, determines whether the identity information of expressive features is known identities information.
By the expressive features in the facial image that extracts, face characteristic corresponding with known identities information compares,
If it does, then can determine that the corresponding identity information of the expressive features is consistent with known identities information, otherwise, it is determined that the expression is special
It levies corresponding identity information and known identities information is inconsistent, that is, can determine whether the corresponding people of the expressive features and known identities information
Whether corresponding people is the same person, such as under the application scenarios that ticket checking is entered the station, can based on the above method extract it is current this
Personal expressive features, are compared with the corresponding face characteristic of known identities information obtained on identity-based certificate, verify
Whether this current people is the corresponding people of known identities information on identity document.
Further, authentication module 306, expressive features and known identities for calculating multiple facial images are believed
Cease the similarity of corresponding face characteristic.
Wherein, the method for calculating the similarity between characteristic point can be real by similarity calculating method in the prior art
The distance between it is existing, for example calculate two characteristic points.
When similarity be greater than preset threshold, determine the identity information of expressive features for known identities information, otherwise, it is determined that table
The identity information of feelings feature is not known identities information.
Further, which further includes fraud expression determination module 307, for foundation Expression Recognition as a result, by expression
Recognition result is matched with the expression in default fraud expression library, determines whether Expression Recognition result is fraud expression.
Wherein, the expression of a large amount of existing frauds of storage in fraud expression library is preset.
Further, it cheats in expression determination module 307, if Expression Recognition result is fraud expression, which is also wrapped
Warning module is included, for generating prompting message corresponding with expression is cheated, and prompting message is sent to the terminal of related personnel
Equipment.
Wherein, by the expression being recognized accurately in video in face, it can further accurately identify whether the expression takes advantage of
Expression is cheated, if it is, the generation of fraud can be judged in time and can effectively be prevented.
The scheme recorded in the present embodiment as a result, the expression identified based on Expression Recognition model is as a result, can further sentence
Whether the expression of breaking is fraud expression, so that the program can be widely used under different application scenarios, if it is decided that certain
The expression of people is fraud expression, and further processing can be made to this person, avoids making other frauds.
A kind of expression that the embodiment of the present application one and embodiment two provide, which can be performed, in the expression recognition apparatus of the present embodiment knows
Other method, realization principle is similar, and details are not described herein again.
Embodiment five
The embodiment of the present application provides a kind of Expression Recognition terminal 40, as shown in figure 4, Expression Recognition terminal shown in Fig. 4
40 include: processor 401 and memory 403.Wherein, processor 401 is connected with memory 403, is such as connected by bus 402.
Optionally, Expression Recognition terminal 40 can also include transceiver 404.It should be noted that transceiver 404 is unlimited in practical application
In one, the structure of the Expression Recognition terminal 40 does not constitute the restriction to the embodiment of the present application.
Wherein, processor 401 is applied in the embodiment of the present application, for realizing image determining module 301 shown in Fig. 3,
Characteristic extracting module 302, the function of time series determining module 303 and Expression Recognition module 304.Transceiver 404 includes connecing
Receipts machine and transmitter.
Processor 401 can be CPU, general processor, DSP, ASIC, FPGA or other programmable logic device, crystalline substance
Body pipe logical device, hardware component or any combination thereof.It, which may be implemented or executes, combines described by present disclosure
Various illustrative logic blocks, module and circuit.Processor 401 is also possible to realize the combination of computing function, such as wraps
It is combined containing one or more microprocessors, DSP and the combination of microprocessor etc..
Bus 402 may include an access, and information is transmitted between said modules.Bus 402 can be pci bus or EISA
Bus etc..Bus 402 can be divided into address bus, data/address bus, control bus etc..For convenient for indicating, in Fig. 4 only with one slightly
Line indicates, it is not intended that an only bus or a type of bus.
Memory 403 can be ROM or can store the other kinds of static storage device of static information and instruction, RAM
Or the other kinds of dynamic memory of information and instruction can be stored, it is also possible to EEPROM, CD-ROM or other CDs
Storage, optical disc storage (including compression optical disc, laser disc, optical disc, Digital Versatile Disc, Blu-ray Disc etc.), magnetic disk storage medium
Or other magnetic storage apparatus or can be used in carry or store have instruction or data structure form desired program generation
Code and can by any other medium of computer access, but not limited to this.
Optionally, memory 403 be used for store execution application scheme application code, and by processor 401
Control executes.Processor 401 is for executing the application code stored in memory 403, to realize that embodiment illustrated in fig. 3 mentions
The movement of the expression recognition apparatus 30 of confession.
A kind of Expression Recognition terminal provided by the embodiments of the present application, compared with prior art, application scheme passes through determination
Multiple facial images continuous in time of same face in video to be identified；The expression for extracting multiple facial images respectively is special
Sign, determines expressive features time series according to the sequential relationship of multiple facial images and each expressive features；And then it is expression is special
Sign time series is input to preset Expression Recognition model, determines expression classification corresponding with expressive features time series, and will
The corresponding expression of expression classification is as Expression Recognition result；In above scheme, expression is established based on expressive features time series and is known
Other model identifies the expression in video by the Expression Recognition model, since expressive features time series can react
Out in the video expression timing feature, therefore may recognize that based on expressive features time series the subtle change of expression in video
Change, so that Expression Recognition result is more acurrate, and due to the slight change of expression, so that identifiable expression classification is richer, because
This Expression Recognition model based on the training of timing expression characteristic sequence may recognize that the expression of more multiple expression classification.
A kind of Installation practice of the Expression Recognition terminal provided by the embodiments of the present application suitable for above-described embodiment four, and
With inventive concept identical with above-mentioned apparatus example IV and identical beneficial effect, details are not described herein.
Embodiment six
The embodiment of the present application provides a kind of computer readable storage medium, which is stored at least one finger
It enables, at least one section of program, code set or instruction set, at least one instruction, at least one section of program, code set or instruction set are by handling
Device is loaded and is executed to realize method shown in embodiment one.
The embodiment of the present application provides a kind of computer readable storage medium, and compared with prior art, application scheme is logical
Cross the multiple facial images continuous in time for determining same face in video to be identified；Multiple facial images are extracted respectively
Expressive features；Expressive features time series is determined according to the sequential relationship of multiple facial images and each expressive features；And then will
Expressive features time series is input to preset Expression Recognition model, determines expression class corresponding with expressive features time series
Not, and using the corresponding expression of expression classification as Expression Recognition result；In above scheme, established based on expressive features time series
Expression Recognition model identifies the expression in video by the Expression Recognition model, due to expressive features time series
The timing feature of expression in the video can be reflected, therefore expression in video may recognize that based on expressive features time series
Slight change, so that Expression Recognition result is more acurrate, and due to the slight change of expression, so that identifiable expression classification is richer
Richness, therefore the Expression Recognition model based on the training of timing expression characteristic sequence may recognize that the expression of more multiple expression classification.
The embodiment of the present application provides a kind of computer readable storage medium, which is stored at least one finger
It enables, at least one section of program, code set or instruction set, at least one instruction, at least one section of program, code set or instruction set are by handling
Device is loaded and is executed to realize method shown in embodiment two.Details are not described herein.
It should be understood that although each step in the flow chart of attached drawing is successively shown according to the instruction of arrow,
These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps
Execution there is no stringent sequences to limit, can execute in the other order.Moreover, at least one in the flow chart of attached drawing
Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps
Completion is executed, but can be executed at different times, execution sequence, which is also not necessarily, successively to be carried out, but can be with other
At least part of the sub-step or stage of step or other steps executes in turn or alternately.
The above is only some embodiments of the application, it is noted that those skilled in the art are come
It says, under the premise of not departing from the application principle, several improvements and modifications can also be made, these improvements and modifications also should be regarded as
The protection scope of the application.
Claims (10)
1. a kind of expression recognition method characterized by comprising
Determine multiple facial images continuous in time of same face in video to be identified；
The expressive features of the multiple facial image are extracted respectively；
Expressive features time series is determined according to the sequential relationship of the multiple facial image and each expressive features；
The expressive features time series is input to preset Expression Recognition model, the determining and expressive features time series
Corresponding expression classification, and using the corresponding expression of the expression classification as Expression Recognition result.
2. expression recognition method according to claim 1, which is characterized in that the training of the preset Expression Recognition model
Method specifically includes:
Determine multiple facial images of the corresponding Time Continuous of expression video in training set；
Extract the expressive features in each facial image；
By the expressive features of each facial image according to sequential combination at expressive features time series；
Expressive features time series and expression classification to mark in advance are input to Recognition with Recurrent Neural Network mould as training sample
Type carries out feature training to the Recognition with Recurrent Neural Network model, until convergence, obtains Expression Recognition model.
3. expression recognition method according to claim 2, which is characterized in that it is described to the Recognition with Recurrent Neural Network model into
The training of row feature, until convergence, obtains Expression Recognition model, specifically includes:
Determine that the training sample belongs to the probability value of each expression classification；
Using the corresponding expression of expression classification of maximum probability value as output result；
The phase that the output result and the training sample are demarcated in advance by the loss function in Recognition with Recurrent Neural Network model
Classification results are hoped to be compared, when the output result of the loss function is less than predetermined threshold, the training of the training sample
Terminate；
Otherwise, the training sample is re-entered into Recognition with Recurrent Neural Network model and is trained, led to before re -training
Inverse algorithms are crossed, each weight of Recognition with Recurrent Neural Network model is adjusted.
4. expression recognition method according to claim 2, which is characterized in that expression video in the training set with it is corresponding
Expression classification stored in the form of two-dimentional table structure.
5. expression recognition method according to claim 1, which is characterized in that the method also includes:
The expressive features of the multiple facial image face characteristic corresponding with known identities information is compared, determines the expression
Whether the identity information of feature is known identities information.
6. expression recognition method according to claim 1, which is characterized in that the method also includes:
According to the Expression Recognition as a result, the Expression Recognition result is matched with default fraud expression library, determine
Whether the Expression Recognition result is fraud expression.
7. expression recognition method according to claim 6, which is characterized in that if the Expression Recognition result is fraud table
Feelings, the method also includes:
Prompting message corresponding with the fraud expression is generated, and the terminal that the prompting message is sent to related personnel is set
It is standby.
8. a kind of expression recognition apparatus characterized by comprising
Image determining module, for determining multiple facial images continuous in time of same face in video to be identified；
Characteristic extracting module, for extracting the expressive features of the multiple facial image respectively；
Time series determining module determines table for the sequential relationship and each expressive features according to the multiple facial image
Feelings feature time series；
Expression Recognition module, for the expressive features time series to be input to preset Expression Recognition model, determining and institute
The corresponding expression classification of expressive features time series is stated, and using the corresponding expression of the expression classification as Expression Recognition result.
9. a kind of Expression Recognition terminal characterized by comprising
Processor, memory and bus；
The bus, for connecting the processor and the memory；
The memory, for storing operational order；
The processor, for executing the described in any item methods of the claims 1 to 7 by calling the operational order.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the storage medium is deposited
Contain at least one instruction, at least one section of program, code set or instruction set, at least one instruction, at least one section of journey
Sequence, the code set or instruction set are loaded by the processor and are executed to realize side as described in any one of claim 1 to 7
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810990197.6A CN109325422A (en) | 2018-08-28 | 2018-08-28 | Expression recognition method, device, terminal and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810990197.6A CN109325422A (en) | 2018-08-28 | 2018-08-28 | Expression recognition method, device, terminal and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109325422A true CN109325422A (en) | 2019-02-12 |
Family
ID=65264020
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810990197.6A Withdrawn CN109325422A (en) | 2018-08-28 | 2018-08-28 | Expression recognition method, device, terminal and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109325422A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111523501A (en) * | 2020-04-27 | 2020-08-11 | 阳光保险集团股份有限公司 | Body mass index prediction method and device |
| WO2020238552A1 (en) * | 2019-05-30 | 2020-12-03 | 深圳壹账通智能科技有限公司 | Micro-expression identification-based examination instruction generation method, device, and computer apparatus |
| CN112381019A (en) * | 2020-11-19 | 2021-02-19 | 平安科技（深圳）有限公司 | Compound expression recognition method and device, terminal equipment and storage medium |
| WO2021068613A1 (en) * | 2019-10-12 | 2021-04-15 | 深圳壹账通智能科技有限公司 | Face recognition method and apparatus, device and computer-readable storage medium |
| WO2021088556A1 (en) * | 2019-11-05 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, device, and storage medium |
| CN112990016A (en) * | 2021-03-16 | 2021-06-18 | 中国平安人寿保险股份有限公司 | Expression feature extraction method and device, computer equipment and storage medium |
| CN113283978A (en) * | 2021-05-06 | 2021-08-20 | 北京思图场景数据科技服务有限公司 | Financial risk assessment method based on biological basis, behavior characteristics and business characteristics |
| WO2021164466A1 (en) * | 2020-02-20 | 2021-08-26 | 深圳壹账通智能科技有限公司 | Method and system for analyzing user information |
| CN113283978B (en) * | 2021-05-06 | 2024-05-10 | 北京思图场景数据科技服务有限公司 | Financial risk assessment method based on biological basis, behavioral characteristics and business characteristics |
-
2018
- 2018-08-28 CN CN201810990197.6A patent/CN109325422A/en not_active Withdrawn
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020238552A1 (en) * | 2019-05-30 | 2020-12-03 | 深圳壹账通智能科技有限公司 | Micro-expression identification-based examination instruction generation method, device, and computer apparatus |
| WO2021068613A1 (en) * | 2019-10-12 | 2021-04-15 | 深圳壹账通智能科技有限公司 | Face recognition method and apparatus, device and computer-readable storage medium |
| WO2021088556A1 (en) * | 2019-11-05 | 2021-05-14 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, device, and storage medium |
| WO2021164466A1 (en) * | 2020-02-20 | 2021-08-26 | 深圳壹账通智能科技有限公司 | Method and system for analyzing user information |
| CN111523501B (en) * | 2020-04-27 | 2023-09-15 | 阳光保险集团股份有限公司 | Body mass index prediction method and device |
| CN111523501A (en) * | 2020-04-27 | 2020-08-11 | 阳光保险集团股份有限公司 | Body mass index prediction method and device |
| WO2022105130A1 (en) * | 2020-11-19 | 2022-05-27 | 平安科技（深圳）有限公司 | Compound expression recognition method, device, terminal apparatus, and storage medium |
| CN112381019B (en) * | 2020-11-19 | 2021-11-09 | 平安科技（深圳）有限公司 | Compound expression recognition method and device, terminal equipment and storage medium |
| CN112381019A (en) * | 2020-11-19 | 2021-02-19 | 平安科技（深圳）有限公司 | Compound expression recognition method and device, terminal equipment and storage medium |
| CN112990016A (en) * | 2021-03-16 | 2021-06-18 | 中国平安人寿保险股份有限公司 | Expression feature extraction method and device, computer equipment and storage medium |
| CN112990016B (en) * | 2021-03-16 | 2022-09-02 | 中国平安人寿保险股份有限公司 | Expression feature extraction method and device, computer equipment and storage medium |
| CN113283978A (en) * | 2021-05-06 | 2021-08-20 | 北京思图场景数据科技服务有限公司 | Financial risk assessment method based on biological basis, behavior characteristics and business characteristics |
| CN113283978B (en) * | 2021-05-06 | 2024-05-10 | 北京思图场景数据科技服务有限公司 | Financial risk assessment method based on biological basis, behavioral characteristics and business characteristics |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109325422A (en) | Expression recognition method, device, terminal and computer readable storage medium | |
| EP3477519A1 (en) | Identity authentication method, terminal device, and computer-readable storage medium | |
| CN106776936A (en) | intelligent interactive method and system | |
| CN108429619A (en) | Identity identifying method and system | |
| WO2020034750A1 (en) | Classification model generation method and device, and data identification method and device | |
| CN107679997A (en) | Method, apparatus, terminal device and storage medium are refused to pay in medical treatment Claims Resolution | |
| CN108090351A (en) | For handling the method and apparatus of request message | |
| CN107679082A (en) | Question and answer searching method, device and electronic equipment | |
| CN114610773A (en) | Data application scene recognition method based on big data and cloud computing server | |
| CN110851761A (en) | Infringement detection method, device and equipment based on block chain and storage medium | |
| CN110197389A (en) | A kind of user identification method and device | |
| CN114170482B (en) | Document pre-training model training method, device, equipment and medium | |
| US20200218772A1 (en) | Method and apparatus for dynamically identifying a user of an account for posting images | |
| CN111488813B (en) | Video emotion marking method and device, electronic equipment and storage medium | |
| CN108075962A (en) | The scene recognition method of instant messaging dialogue, apparatus and system | |
| CN111062019A (en) | User attack detection method and device and electronic equipment | |
| CN113705792A (en) | Personalized recommendation method, device, equipment and medium based on deep learning model | |
| CN113743522A (en) | Detection method and device for illegal behavior and electronic equipment | |
| US11900336B2 (en) | Computer-based systems configured for automated activity verification based on optical character recognition models and methods of use thereof | |
| CN107786980A (en) | A kind of fraud information recognition methods and its device, mobile terminal, server | |
| CN109918552A (en) | Malice group tells tendency personal identification method, server and computer readable storage medium | |
| CN113362169A (en) | Catalytic recovery optimization method and device | |
| CN109615458A (en) | Client management method, device, terminal device and computer readable storage medium | |
| CN112328793A (en) | Comment text data processing method and device and storage medium | |
| CN112328937A (en) | Information delivery method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40002600 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190212 |