AU2020102556A4 - Psychological state analysis method based on facial micro-expression - Google Patents

Abstract

Our project can introduce self-supervising data of images rotating and spatial attention mechanism. The project can further enhance the recognition accuracy of micro expression recognition, based on optical flow method for data augmentation. To be specific, in MGEC2019 fusion data set, both weightless F-i score and weightless average recall rate can reach up to 80%. 1

Description

DESCRIPTION TITLE

Psychological state analysis method based on facial micro-expression

FIELD OF THE INVENTION

The present invention relates to Psychological state analysis method

based on facial micro-expression.

BACKGROUND

Facial expression can express people's psychological changes clearly,

so it has great potential in personal use, commercial use and government

application. For example, at the exit of the subway, thousands of people

enter and exit every day. Although relevant personnel will carry out

security checks one by one, they are not willing to exclude all high-risk

personnel, who will endanger the safety of others. Therefore, it is very

necessary to have an instrument that can analyze human psychological

state based on facial micro expression to help security inspection. Such

equipment can quickly judge whether it has the intention of attack

according to the micro expression, so as to better protect the national

security. Secondly, in the field of education, teachers can improve the

classroom efficiency according to the similar equipment, and

appropriately mobilize students according to the psychological state of students judged by the equipment, so as to fully improve students' learning enthusiasm. In addition, in medicine, doctors can understand the real psychology of patients according to the micro expression, so as to suit the medicine to the case and improve the treatment efficiency.

This research project is based on the mapping relationship between a

large number of micro expressions and mental states. Micro expression is

a kind of human behavior information, which can be collected by external

devices. Then intelligent machines will judge their mental state through

the corresponding mapping relationship. Human micro expression often

reveals some real emotions that are trying to hide. It is a very effective

non-verbal clue, which can make it a window that I don't understand the

processing process of human real emotions and internal emotions. For

example, when human beings are in fear and worry, their micro

expressions are often everted nostrils and tight lips; when they are in self

blame and shame, the corners of their mouths droop and their chin rises.

However, mental state is also related to many other factors, this topic will

be based on the algorithm used to solve.

The research objectives of unsupervised learning is to learn a robust

image feature representation through self-supervised labels using

abundant unsupervised data.One of the major issues in the micro-expression recognition task is that the amount of data is small.

Therefore, the introduction of unsupervised learning tasks to enhance the

feature extraction learning will play an auxiliary role in the recognition of

micro-expressions.In addition, unsupervised learning does not introduce

new annotation data and no new labor costs will be introduced.Rotating

the optical flow image at some fixed angles does not change the

micro-expression category. Therefore, some fixed rotation angles are

designed as a self-supervised label and added to the model

training.Secondly, because the recognition of micro expressions is based

on some specific positions, such as eyes and mouth, the introduction of a

spatial attention mechanism to highlight the position that plays a key role

in the recognition of micro expressions will help the recognition effect.

The purpose of this project is to introduce the self-supervised

information and attention mechanism of image rotation, and further

improve the recognition accuracy of micro-expressions in view of the

optical flow method based on data augmentation.Specifically, on the

MGEC2019 fusion data set, both the unweighted F-i score and the

unweighted average recall rate reached 80%.

SUMMARY

There are two solutions for facial expression recognition: methods

based on image and based on space and time. Image-based methods

include LBP methods based on manual local key feature point LBP, and

on capsule network-based convolutional neural network features and so

on. Spacetime-based methods include two-time-scale convolutional

neural networks methods, long-term iterative convolutional networks, and

methods based on optical flow. The optical flow-based method takes the

change of micro-expression as input in the form of optical flow, while the

image-based method only considers the content of the image itself,

ignoring the change information of the micro-expression. So the method

based on optical flow have more advantages than the method based on

image. The current optical flow-based methods mainly include better

feature extraction methods (such as dual inception, shallow ternary flow

convolutional neural network, etc.) and data augmentation methods (such

as small motion amplification, domain adaptation methods, etc.) ). Due to

the small amount of micro-expression data (only 442 samples in the

MGEC2019 integrated data). Hence the optical flow method based on

data augmentation has achieved good results.

DESCRIPTION OF PREFERRED EMBODIMENT

A kind of unsupervised learning algorithm that is widely used is called

K-means algorithm, among which K is the number of clusters that users specified to create. K-means algorithm is start with K random centroids.

The algorithm can calculate distances of every point from centroid. Every

point will be distributed to the nearest cluster centroid. Then repeating the

above process several times until cluster centroids stop changing. The

realization idea is setting K cluster centroid points, classifying according

to the principle of nearest, recalculate centroid, repeating the above

process until there is no change.

The research purpose of unsupervised learning is making use of the

abundant unsupervised data to learn image feature representation of a

robust with the help of self-supervising labels.

The major problem of micro expression recognition task is that the data

volume is small, and therefore introducing unsupervised learning to

enhance the feature extraction learning can assist us with recognition of

micro expressions. Moreover, unsupervised learning don' t introduce new

label data and new labor cost. Rotating the optical flow images at a

constant angle don' t change the category of micro expressions, thus we

can set some constant rotating angles as self-supervising labels and add to

model training. Besides, considering that recognition of micro

expressions is based on a number of specific areas, such as mouse and

eyes, we can introduce spatial attention mechanism. Highlighting the key position of micro expression recognition can help improving the recognition effects.

Our project can introduce self-supervising data of images rotating and

spatial attention mechanism. The project can further enhance the

recognition accuracy of micro expression recognition, based on optical

flow method for data augmentation. To be specific, in MGEC2019 fusion

data set, both weightless F-i score and weightless average recall rate can

reach up to 80%.

Claims (1)

CLAIM

1. Psychological state analysis method based on facial micro-expression,

characterized in that, including: a kind of unsupervised learning

algorithm that is widely used is called K-means algorithm, among

which K is the number of clusters that users specified to create;

K-means algorithm is start with K random centroids; the algorithm

can calculate distances of every point from centroid.