CN115472264A - Non-contact psychological state prediction method - Google Patents

Abstract

The invention discloses a non-contact psychological state prediction method, and specifically relates to the technical field of psychological prediction. The invention classifies and processes different monitored persons and trains a deep learning network model to obtain a prediction model, and then obtains a prediction model through The video image of the monitored person obtained from the monitoring data is extracted and input into the prediction model for prediction, so as to analyze the emotional tendency of the monitored person and realize the non-contact psychological state prediction of the monitored person, which not only improves the credibility of the prediction, but also It will not affect the time for the monitored personnel and the questionnaire distribution and viewing personnel, and will not cause unnecessary psychological burden to the monitored personnel; in addition, the present invention can carry out psychological monitoring on a large number of monitored personnel at the same time through the monitoring data, It can be applied to large-scale places with many people, such as training camps, factories, companies, schools, etc., and plays an important role in monitoring the psychological state of a large number of people.

Description

A Non-contact Mental State Prediction Method

technical field

The present invention relates to the technical field of mental prediction, and more specifically, the present invention relates to a non-contact mental state prediction method.

Background technique

Mental state is one of the basic forms of mental activity, which refers to the complete characteristics of mental activity within a certain period of time. Such as attention, fatigue, tension, relaxation, sadness, joy, etc., it has the characteristics of both psychological processes and individual psychological characteristics. It is both temporary and stable. The background against which mental activity unfolds.

At present, in some colleges and universities or employment environments, due to the difficulty of learning, tight performance or excessive pressure, it is easy to cause problems in the psychological state of the personnel. The psychological problem may become more and more serious, not only affecting studies and work, but also causing harm to the health of the personnel in severe cases. In the prior art, the psychological state of the personnel is generally detected by means of questionnaires, but it is easy to appear The situation of random selection makes the credibility low, and also wastes the time of personnel, and cannot monitor and process by itself. Therefore, it is of great significance to study a non-contact mental state prediction method to solve the above problems.

Contents of the invention

In order to overcome the above-mentioned defects of the prior art, the present invention provides a non-contact mental state prediction method. The technical problem to be solved by the present invention is: the way of detecting the mental state in the prior art has low reliability, And it will also waste the time of personnel, and it is impossible to monitor and deal with the problem by itself.

In order to achieve the above object, the present invention provides the following technical solution: a non-contact mental state prediction method, comprising the following steps:

S1. Acquire the basic information of the monitored personnel, and use the data on the social platform to obtain social data.

S2. Classify the monitored persons according to similarity to obtain different categories.

S3. Obtain precursory behaviors and expression changes of different types of people when they have psychological problems through big data, and train the deep learning network model to obtain a prediction model.

S4. Obtain and collect the current video image of the monitored person through monitoring data acquisition, and extract facial image information and behavior information at the same time.

S5. Extract features from the video data of the monitored person to obtain behavior information, and at the same time identify facial image information, extract facial image features, and judge the expression of the monitored person.

S6. Input behavioral information and expression changes into the prediction model, and the prediction model judges whether there is a corresponding precursory behavior, and judges the current psychological state of the monitored person, analyzes the emotional tendency of the monitored person, and outputs it, so as to realize the psychological state of the monitored person. Prediction of the state, when it is found that there is a problem with the emotional tendency of the monitored person, an early warning will be given to the relevant personnel, and the relevant personnel will be notified to intervene in the psychological state of the monitored person.

As a further solution of the present invention: the basic information includes the name, gender, age, education background, work, family composition and social status of the monitored person.

As a further solution of the present invention: the deep learning network model includes but is not limited to a deep convolutional neural network model built in the tensorflow environment, which includes an input layer, three convolutional layers, two fully connected layers and an output layer, where there are corresponding pooling layers after the first two convolutional layers.

As a further solution of the present invention: the specific method for training the deep learning network model is:

S31. Perform cleaning and dimensionality reduction processing on specific information on precursor behaviors and facial expression changes of different types of personnel obtained through big data when they have psychological problems.

S32. Divide multiple sets of data into training data and test data, wherein 80% of the training data and 20% of the test data are used as the input of the deep learning network model to train the deep learning network model.

S33. After the training is completed, the test is performed on the test data. When the test accuracy rate is higher than 99%, the training is terminated, and the trained deep learning network model is used as a prediction model to predict the corresponding psychological state through different behaviors and facial expressions.

As a further solution of the present invention: the monitored personnel are classified according to similarity, and the specific methods for obtaining different categories are:

S21. Sorting out the basic information of the monitored persons, screening the text, pictures and videos in the social data, selecting data related to emotions, and performing theme analysis.

S22. According to the Big Five personality theory, different monitors are divided into five types of ABCDE, among which, ABCDE respectively represent the five types of cheerful, responsible, extraverted, agreeable and neurotic, and adopt the principle of the Big Five personality scale , quantify the types of different monitored personnel into a vector, namely:

P=<A _score, B _score, C _score, D _score, E _score> .

S23. Show some characteristics of each type more specifically, and classify these characteristics to obtain x sub-dimensions, and further calculate the score of each sub-dimension of the monitored person on each personality, that is

Among them, e _i represents the sub-dimension of extraversion, and _ki is the influence weight of each dimension on personality type, which is initialized to 1/n, and then optimized and adjusted according to the feedback of the monitored personnel.

S24. Express the characteristics of each type of personality into emotions and behaviors, construct the logical relationship between personality, emotions and behaviors, and make logical connections. At the same time, from the three relationship types of infection, expression, intervention and embodiment, Construct a series of meta-structures to calculate the subdimension scores of each personality type of the monitored person, and then obtain the tendency score of the personality type of the monitored person, so as to judge the current psychological state of the monitored person.

As a further solution of the present invention: the psychological state is divided into four types: healthy state, bad state, psychological disorder and psychological crisis.

As a further solution of the present invention: the calculation formula of each personality type sub-dimension score is:

Among them, n _m is the number of emotion types, n _b is the number of behavior types, M=(m1,m2,...,m _nm ), B=(b1,b2,...,b _nb ), I _nm ×n _b is The relationship matrix between emotion type and behavior type, if there is a link relationship, it is 1, otherwise it is 0.

As a further solution of the present invention: the described text picture and video in the social data are screened, and the specific method of selecting data related to emotion is: a large amount of text data obtained is carried out text embedding and Kmeans clustering through the Bert model to identify sentences close to the centroid for automatic summary selection to select specific data for sentiment relevance.

As a further solution of the present invention: the described facial image information is identified, extracting facial image features, and judging the expression of the monitored personnel specifically includes the following steps:

S51. Extract the face part in the monitoring data, perform grayscale processing to obtain the expression image, and perform preprocessing, specifically: cut and normalize the size according to the characteristics of the three courts and five eyes of the face and the set model of the face to remove areas that have nothing to do with expressions, and reduce the interference of irrelevant information on expression recognition;

S52, then carry out multi-feature extraction to the facial expression image after pretreatment, three kinds of features extracted are respectively LDP, DWT and Sobel, input three kinds of feature maps in the convolutional neural network in the form of three channels, carry out self-adaptive fusion, Finally, the expression classification is performed on the fused features through the Softmax classifier.

The beneficial effects of the present invention are:

1. The present invention classifies and processes different monitored personnel, and obtains precursory behaviors and expression changes when different types of personnel have psychological problems through big data, trains a deep learning network model, obtains a prediction model, and then monitors The video image of the monitored person obtained from the data is extracted and input into the prediction model for prediction, so as to judge the current psychological state of the monitored person, analyze the emotional tendency of the monitored person, and realize the non-contact psychological state prediction of the monitored person, which not only improves The credibility of the prediction is guaranteed, and it will not affect the time of the monitored personnel and the questionnaire distribution and viewing personnel, and will not cause unnecessary psychological burden to the monitored personnel;

2. The present invention carries out pre-warning processing on the relevant personnel when the psychological state and emotional tendency of the monitored personnel have a sign of psychological problems, and notifies them to carry out psychological intervention on the measured personnel, so that the negative emotions of the personnel can be channeled in a timely manner ;

3. The present invention can carry out psychological monitoring on a large number of monitored personnel at the same time through monitoring data, and is applicable to large-scale places with many personnel such as training camps, factories, companies, schools, etc., and plays an important role in monitoring the psychological state of a large number of personnel. significance.

Description of drawings

Fig. 1 is the schematic flow chart of mental state prediction of the present invention;

Fig. 2 is a schematic flow chart of the deep learning network model training method of the present invention;

Fig. 3 is a schematic flow chart of the inventor's classification.

detailed description

The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Example:

As shown in Figure 1-3, a non-contact mental state prediction method includes the following steps:

S1. Obtain basic information such as the name, gender, age, education, work, family composition, and social situation of the monitored personnel through the management personnel, and at the same time express emotions with high probability from Weibo, Douban movie reviews, circle of friends, and NetEase cloud music reviews Crawl massive amounts of published data from the platform.

S2. Organize the basic information of the monitored personnel, and screen the text pictures and videos in the social data, select the data related to emotions, and conduct theme analysis, and use the Bert model to text the large amount of text data obtained. Embedding and Kmeans clustering to identify sentences near centroids for automatic summary selection to select specific data for sentiment relevance Text embedding and Kmeans clustering via Bert model to identify sentences near centroids for automatic summary selection Important sentences can be automatically screened out from a large number of social texts of the monitored personnel, which can better solve the problems of information fragmentation and useless information interference, and improve the efficiency and accuracy of the model.

According to the Big Five Personality Theory, different monitors are divided into five types of ABCDE, among which, ABCDE respectively represent the five types of cheerful, responsible, extroverted, agreeable and neurotic, and adopt the principle of the Big Five Personality Scale. The types of different monitored persons are quantified as a vector, namely:

P=<A _score, B _score, C _score, D _score, E _score> .

Then each type shows some characteristics more specifically, and classify these characteristics to obtain x sub-dimensions, and further calculate the score of each sub-dimension of the monitored person on each personality, that is

Among them, e _i represents the sub-dimension of extraversion, and _ki is the influence weight of each dimension on personality type, which is initialized to 1/n, and then optimized and adjusted according to the feedback of the monitored personnel.

Since emotions will affect behavior to a certain extent, and behavior is often an expression of emotion, so the characteristics of each type of personality are expressed in emotion and behavior, and the logical relationship between personality, emotion and behavior is constructed, and Make logical connections; for example, cheerfulness indicates the amount and intensity of human interaction, the need for stimulation and the ability to enjoy pleasure, this dimension combines social, active, personally oriented individuals with silent, serious, shy Compared with quiet people, this aspect can be measured by two qualities: the level of interpersonal involvement and the level of vitality. An important emotional expression of extraversion, needing companionship is an important behavioral expression of extraversion, so being happy and seeking companionship both reflect the extraverted personality tendency of the person being monitored, and behavior and emotion usually affect each other, but This effect is affected by personality types. People with different personality tendencies express different emotions and behavioral connections. The expression intervention of extraversion on companionship-seeking behavior points to happiness, because extraversion usually likes to be in contact with people, and is enthusiastic and talkative. When they are in a group, they usually express a positive emotion, such as happiness; vice versa, it is the same from emotion to behavior. At the same time, a series of meta-structures are constructed from the three relationship types of infection, expression, intervention and embodiment It is used to calculate the subdimension scores of each personality type of the monitored person, and then obtain the tendency score of the personality type of the monitored person, so as to judge the current psychological state of the monitored person.

S3. Obtain the precursory behavior and expression changes of different types of people when they have psychological problems through big data, and perform cleaning and dimensionality reduction processing on the specific information of precursory behaviors and facial expressions when different types of people have psychological problems obtained through big data. .

Multiple sets of data are divided into training data and test data, wherein the training data is 80%, and the test data is 20%. The training data is used as the input of the deep learning network model to train the deep learning network model.

After the training is completed, the test data is used to test. When the test accuracy is higher than 99%, the training ends, and the trained deep learning network model is used as a prediction model to predict the corresponding mental state through different behaviors and facial expressions.

S4. Obtain and collect the current video image of the monitored person through monitoring data acquisition, and extract facial image information and behavior information at the same time.

S5. Perform feature extraction on the video data of the monitored person to obtain behavior information, and at the same time identify the facial image information, extract facial image features, and judge the expression of the monitored person, specifically: according to the characteristics of the three courts and five eyes of the face Crop and normalize the size of the set model of the face to remove areas that are not related to expressions, and reduce the interference of irrelevant information on expression recognition;

Then, multi-feature extraction is performed on the preprocessed expression image. The three extracted features are LDP, DWT and Sobel. The three feature maps are input into the convolutional neural network in the form of three channels for adaptive fusion. Finally, through The Softmax classifier performs expression classification on the fused features.

S6. Input behavioral information and expression changes into the prediction model, and the prediction model judges whether there is a corresponding precursory behavior, and judges the current psychological state of the monitored person, analyzes the emotional tendency of the monitored person, and outputs it, so as to realize the psychological state of the monitored person. Prediction of the state, when it is found that there is a problem with the emotional tendency of the monitored person, an early warning will be given to the relevant personnel, and the relevant personnel will be notified to intervene in the psychological state of the monitored person.

Deep learning network models include but are not limited to deep convolutional neural network models built in the tensorflow environment, which include an input layer, three convolutional layers, two fully connected layers and an output layer, where the first two convolutional Each layer has a corresponding pooling layer.

Mental state is divided into four types: healthy state, bad state, psychological disorder and psychological crisis.

The formula for calculating the subdimension scores of each personality type is:

Among them, n _m is the number of emotion types, n _b is the number of behavior types, M=(m1,m2,...,m _nm ), B=(b1,b2,...,b _nb ), I _nm ×n _b is The relationship matrix between emotion type and behavior type, if there is a link relationship, it is 1, otherwise it is 0.

In the embodiment of the present invention, the mental state prediction method classifies and processes different monitored persons, and obtains precursory behaviors and expression changes when different types of persons have psychological problems through big data, and trains the deep learning network model , get the prediction model, and then use the video image of the monitored person obtained from the monitoring data to perform feature extraction and input it into the prediction model for prediction, so as to judge the current psychological state of the monitored person, analyze the emotional tendency of the monitored person, and realize the monitoring of the monitored person. Noncontact Mental State Prediction.

Some points that should be explained at last are: although, the present invention has been described in detail with general explanation and specific embodiment above, but on the basis of the present invention, above each embodiment is only in order to illustrate technical scheme of the present invention , but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions described in the foregoing embodiments, or modify some of them Or all technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (9)

1. A non-contact mental state prediction method, characterized in that, comprising the following steps: S1. Obtain the basic information of the monitored personnel, and use the data on the social platform to obtain social data; S2. Classify the monitored personnel according to the similarity to obtain different categories; S3. Obtain precursory behaviors and expression changes of different types of people when they have psychological problems through big data, and train the deep learning network model to obtain a prediction model; S4. Obtain and collect the current video image of the monitored person through monitoring data, and extract facial image information and behavior information at the same time; S5. Perform feature extraction on the video data of the monitored person to obtain behavior information, and at the same time identify the facial image information, extract facial image features, and judge the expression of the monitored person; S6. Input behavioral information and expression changes into the prediction model, and the prediction model judges whether there is a corresponding precursory behavior, and judges the current psychological state of the monitored person, analyzes the emotional tendency of the monitored person, and outputs it, so as to realize the psychological state of the monitored person. Prediction of the state, when it is found that there is a problem with the emotional tendency of the monitored person, an early warning will be given to the relevant personnel, and the relevant personnel will be notified to intervene in the psychological state of the monitored person. 2. A non-contact mental state prediction method according to claim 1, characterized in that: the basic information includes the name, gender, age, education, work, family composition and social status of the monitored person. 3. A kind of non-contact mental state prediction method according to claim 1, characterized in that: said deep learning network model includes but not limited to a deep convolutional neural network model built in tensorflow environment, which includes An input layer, three convolutional layers, two fully connected layers and an output layer, where there are corresponding pooling layers after the first two convolutional layers. 4. a kind of non-contact mental state prediction method according to claim 3, is characterized in that: the described concrete method that deep learning network model is trained is: S31. Perform cleaning and dimensionality reduction processing on the specific information on precursory behaviors and facial expression changes of different types of personnel obtained through big data when they have psychological problems; S32. Divide multiple groups of data into training data and test data, wherein the training data is 80%, and the test data is 20%. The training data is used as the input of the deep learning network model, and the deep learning network model is trained; S33. After the training is completed, the test is performed on the test data. When the test accuracy rate is higher than 99%, the training is terminated, and the trained deep learning network model is used as a prediction model to predict the corresponding psychological state through different behaviors and facial expressions. 5. A kind of non-contact mental state prediction method according to claim 1, characterized in that: said monitored personnel are classified according to similarity, and the specific methods for obtaining different categories are: S21. Sorting out the basic information of the monitored personnel, screening the text, pictures and videos in the social data, selecting data related to emotions, and performing theme analysis; S22. According to the Big Five personality theory, different monitors are divided into five types of ABCDE, among which, ABCDE respectively represent the five types of cheerful, responsible, extraverted, agreeable and neurotic, and adopt the principle of the Big Five personality scale , quantify the types of different monitored personnel into a vector, namely: P＝<A _score, B _score, C _score, D _score, E _score> ; S23. Show some characteristics of each type more specifically, and classify these characteristics to obtain x sub-dimensions, and further calculate the score of each sub-dimension of the monitored person on each personality, that is

Among them, e _i represents the sub-dimension of extraversion, and _ki is the influence weight of each dimension on personality type, which is initialized to 1/n, and then optimized and adjusted according to the feedback of the monitored personnel; S24. Express the characteristics of each type of personality into emotions and behaviors, construct the logical relationship between personality, emotions and behaviors, and make logical connections. At the same time, from the three relationship types of infection, expression, intervention and embodiment, Construct a series of meta-structures to calculate the subdimension scores of each personality type of the monitored person, and then obtain the tendency score of the personality type of the monitored person, so as to judge the current psychological state of the monitored person. 6. A non-contact mental state prediction method according to claim 5, characterized in that: said mental state is divided into four types: healthy state, bad state, psychological disorder and psychological crisis. 7. A kind of non-contact mental state prediction method according to claim 5, characterized in that: the calculation formula of each personality type sub-dimension score is:

Among them, n _m is the number of emotion types, n _b is the number of behavior types, M=(m1,m2,...,m _nm ), B=(b1,b2,...,b _nb ), I _nm ×n _b is The relationship matrix between emotion type and behavior type, if there is a link relationship, it is 1, otherwise it is 0. 8. A kind of non-contact mental state prediction method according to claim 5, characterized in that: said text pictures and videos in the social data are screened, and the specific method for selecting data related to emotions is: A large amount of text data will be acquired through the Bert model for text embedding and Kmeans clustering to identify sentences close to the centroid for automatic summary selection, so as to select specific data related to emotion. 9. A kind of non-contact mental state prediction method according to claim 1, characterized in that: said facial image information is identified, extracting facial image features, and judging the expression of the monitored personnel specifically includes the following steps: S51. Extract the face part in the monitoring data, perform grayscale processing to obtain the expression image, and perform preprocessing, specifically: cut and normalize the size according to the characteristics of the three courts and five eyes of the face and the set model of the face to remove areas that have nothing to do with expressions, and reduce the interference of irrelevant information on expression recognition; S52, then carry out multi-feature extraction to the facial expression image after pretreatment, three kinds of features extracted are respectively LDP, DWT and Sobel, input three kinds of feature maps in the convolutional neural network in the form of three channels, carry out self-adaptive fusion, Finally, the expression classification is performed on the fused features through the Softmax classifier.

Images