CN113420591B - Emotion-based OCC-PAD-OCEAN federal cognitive modeling method - Google Patents

Abstract

The invention provides an emotion-based OCC-PAD-OCEAN federal cognitive modeling method, which comprises the following steps: constructing a VGG-FACS-OCC model, and calculating the emotion space vector of the tested video; according to the parameter quantization mapping relation between the OCC emotion space and the PAD emotion space in the OCC-PAD-OCEAN model, mapping the emotion space vector to the PAD emotion space to obtain an emotion space vector; and mapping the mood space vector to an OCEAN personality space to obtain a personality space vector. According to the invention, the expression is mapped to the PAD mood space through the established expression-mood mapping relation, and then the average mood in a period of time is mapped through the established mood-personality mapping relation, so that extraction of personality characteristics is realized, and finally, information with certain statistical significance and credibility in the personality space is obtained.

Description

A sentiment-based OCC-PAD-OCEAN federated cognitive modeling method

technical field

The invention relates to the technical field of psychological cognitive modeling, in particular to an emotion-based OCC-PAD-OCEAN federated cognitive modeling method.

Background technique

Psychology is an experimental science, and the vast majority of psychological research progress is based on the psychological experimental paradigm, which collects and analyzes the subjective and objective data of the subjects. In the experiment, the main tester is required to observe and record all aspects of the behavior of the testee under the condition of strict control of the variables, and analyze the collected data results. Even so, there are still many problems in psychological experiments, such as low reliability and validity of experimental results, and unrepeatable experiments. Some of these problems come from the bystander effect, and some of them come from the limitations of laboratory experiments, etc., which lead to psychological experiments being limited to certain scenarios and unable to be extrapolated and extended. In response to the above problems, computer technology can play a role in precise control and quantification of data. This modern psychometric technique can have many benefits, including avoiding complex reliability measurements, improving construct validity, avoiding exposure effects, and high measurement efficiency. The main purpose of most psychological experiments is to explore the principles of human behavior or human cognitive models: based on the perspective of data collection, the observation data is collected through computer technology, which can realize precise digital control of the entire experimental environment, such as accurate collection of video signals, Audio signals, sensor data, human motion information, etc. Based on the perspective of constructing the experimental environment, computer assistance can enable the subjects to have an immersive experience. For example, the use of virtual reality technology (VR) in the study of emotional psychology can make emotions more visible than general pictures or language stimuli. induced more effectively. In addition, computer technology can simulate hypothetical models to explain the observed behavior, and in the case of limited experimental conditions, computer simulation can also perform preliminary idealized verification of hypotheses. In the past decade, the study of human emotional behavior has attracted more and more attention. Affective computing is an interdisciplinary subject based on psychology and computer science, which studies how to use computers to recognize, model and even express human emotions. Personality computing, which extends from emotional computing, can advance all technologies related to the understanding and prediction of human behavior.

In psychological research on human behavior, prediction, etc., personality is a very important determinant, which can describe stable personal characteristics, which can usually be measured in a quantitative way, explain and predict observable behavior. difference. The Big Five Personality Model (FFM) is an important theory in personality psychology today and one of the most influential models in psychological research. Its five factors include openness, conscientiousness, extraversion, agreeableness, and neuroticism. Social psychologist Harry Reis described FFM as "the most scientifically rigorous taxonomy in the behavioral sciences." The Big Five personality model provides a structure for classifying the personality traits of most people, and describes most individual differences concisely and comprehensively through a set of highly replicable dimensions. From a computer point of view, the feature model represents personality in numerical form, which is suitable for computer processing. However, most current personality assessments take the form of self-reports, assessing personality through statements or adjectives on a scale. The self-report paradigm is simple and clear, but it cannot control the authenticity of the subjects' answers. The experimental results are also affected by many irrelevant factors and are prone to large deviations. One of the significant limitations of self-assessment is also that subjects may tend to bias ratings toward social expectations, especially when assessments may have negative consequences, and subjects may hide negative characteristics, resulting in results that do not correspond to real personalities.

Generally speaking, although some cross-innovation research is promoting the calculation and quantification of psychological theories, psychological theories are still based on traditional qualitative conclusions, and it is difficult to provide direct quantitative model support for computer algorithm realization. In addition, computer algorithm programs cannot accurately express the emotion theory and emotion model in psychology, and there was a big barrier between the two. Most of the existing research is usually only considered from either computer science or psychology perspective, rather than from a cross-integration perspective. At the same time, although the facial expression recognition technology based on deep learning has been relatively perfect, the research on using deep learning technology to process psychological signals is still in its infancy. Therefore, starting from the basic theory of emotional psychology, cognitive modeling methods that integrate deep learning and other algorithms for deep integration are still relatively lacking. How to efficiently deal with cognitive problems while improving the interpretability of the model is also a key issue.

Contents of the invention

The invention provides an emotion-based OCC-PAD-OCEAN federated cognitive modeling method that can use deep learning technology to process psychological signals, and solves the problem of deep-level fusion of deep learning algorithms in the prior art. The lack of modeling methods.

In order to solve the above-mentioned technical problems, the method that the present invention provides comprises: S1 builds VGG-FACS-OCC model, calculates the emotional space vector of tested video; Parameters quantify the mapping relationship, and map the emotion space vector to the PAD mood space to obtain the mood space vector; S3 maps the mood space vector to the OCEAN personality space, and obtains the personality space vector through the mood space vector.

Specifically, the S1 specifically includes the following steps: S11 divides the video under test into several picture frames Image _t according to time, and samples the picture frame Image _t at a fixed frequency to obtain a number of sampling frames I _i , (i=1, 2 , 3, ..., n); S12 preprocesses the sampled frame I _i to remove interference information; S13 maps the preprocessed sampled frame I _i to the OCC emotion space to obtain an emotion space vector E _i .

Further, the S12 is to use the preprocessing function Pre to preprocess the sampled frame _Ii , and the S12 specifically includes: S121 uses the MTCNN face recognition algorithm to perform face target detection on the sampled frame _Ii to obtain the target frame set B ={b ₁ , b ₂ ,...b _m }, where, b _i =(xi _, y _i , _hi , w _i , p _i ), xi represents the abscissa of the upper left corner of the target frame, and _{y i represents} the target The ordinate of the upper left corner of the frame, h _i represents the height of the target frame, w _i represents the width of the target frame, p _i represents the confidence of the target frame; S122 determines the height threshold h _t , the width threshold w _t and the confidence threshold p _t ; For any b _i ∈ B, hold B′={h _i ＞h _t and w _i ＞w _t and S123 Obtain the target frame b _* with the highest confidence p _i from B′, crop I _i according to b _* , and adjust the cropped I _i to a specific size to obtain Pre(I _i ), and obtain the emotional space vector E _i =VGG(Pre(I _i )).

Specifically, the S2 includes: S21 calculates the mood space vector M _i =K×E _i , where K is the transformation matrix of the parameter quantization mapping relationship between the OCC emotion space and the PAD mood space; S22 calculates the mapping relationship between the mood space and the personality space Continuation, get M _i =K×E′ _i ; S23 Calculate the average mood space vector

Specifically, the S3 is to calculate the personality space vector P _e =Z×M _v , where Z is the conversion matrix from the PAD mood space to the OCEAN personality space.

The above technical solution has the following advantages or beneficial effects: the present invention maps expressions to the PAD mood space through the established expression-mood mapping relationship, and then maps the average mood within a period of time through the established mood-personality mapping relationship, and utilizes depth Learning technology to process psychological signals, realize the extraction of personality characteristics, and finally obtain information with certain statistical significance and validity in personality space, and solve the cognition of deep integration of algorithms such as deep learning in the existing technology The modeling method is relatively deficient.

Description of drawings

The invention and its characteristics, shapes and advantages will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings. Like numbers designate like parts throughout the drawings. The drawings are not intended to be drawn to scale, emphasis instead being placed upon illustrating the gist of the invention.

Fig. 1 is the brief flowchart of the OCC-PAD-OCEAN federation cognitive modeling method based on emotion that the embodiment of the present invention 1 provides;

Fig. 2 is the model processing schematic diagram of the OCC-PAD-OCEAN federated cognitive modeling method based on emotion that embodiment 1 of the present invention provides;

Fig. 3 is the function expression diagram of each level of VGG-19 provided by Embodiment 1 of the present invention;

Fig. 4 is the FACS-OCC emotion mapping table based on CK+expression feature that embodiment 1 of the present invention provides;

Fig. 5 is the quantitative mapping table of the PAD mood space and emotion provided by Embodiment 1 of the present invention;

Fig. 6 is the data processing procedure schematic diagram of the model of the OCC-PAD-OCEAN federated cognitive modeling method based on emotion that embodiment 1 of the present invention provides;

Fig. 7 is the OCC six emotion dimension time-series data diagrams in all frames of the video of the OCC-PAD-OCEAN federated cognitive modeling method based on emotion provided by Embodiment 1 of the present invention;

Fig. 8 is the timing data diagram of PAD mood space in all frames of the video of the OCC-PAD-OCEAN federated cognitive modeling method based on emotion that the embodiment of the present invention 1 provides;

Fig. 9 is the personality radar map of the output of the OCC-PAD-OCEAN federated cognitive modeling method based on emotion that the embodiment of the present invention 1 provides;

Fig. 10 is the five personality deviation rates of all subjects in the emotion-based OCC-PAD-OCEAN federated cognitive modeling method provided in Embodiment 1 of the present invention.

Detailed ways

The present invention will be further described below in conjunction with accompanying drawing and specific embodiment, but not as limitation of the present invention.

Example 1:

Emotion can reflect a person's short-term state. Due to changes or stimuli in external conditions, emotion can undergo major changes in a short period of time. The OCC emotion model specifies 22 emotion categories as a standard model of emotion synthesis. Based on Ekman's theory that all non-basic emotions can be synthesized by basic emotions, the present invention adopts six basic emotions defined by it to construct the OCC emotion space, i.e. anger ), disgust (disgust), fear (fear), happiness (happiness), sadness (sadness), surprise (surprise), expressed in vector form as E=[e _angry , e _disgust , e _fear , e _happy , e _sad , e _surprise ] ^T , where the value range of each element is [0, 1], indicating the intensity of emotion.

As an intermediate measure of emotion and personality, mood reflects the emotional state of a person over a period of time. From a measurement perspective, mood can be obtained by averaging an individual's affective states over a period of time, however since combinations of discrete affective states (e.g., anger, disgust, fear, happiness, sadness) cannot be meaningfully averaged, a Conceptual system to construct the basic dimension of emotions, so the present invention introduces the PAD mood space composed of mutually independent three-dimensional, that is, Pleasure (P), Arousal (A) and Dominance (D). The form is expressed as M=[m _P , m _A , m _D ] ^T , and the value range of the elements is [-1, 1]. Among them, Pleasure (P) describes the relative advantages of positive emotional state and negative emotional state; Arousal (A) measures the degree to which a person is aroused by "high information" (complex, changing, unexpected) stimuli, and Speed of return to baseline; Dominance (D) assesses a person's sense of control and influence over his or her life circumstances, and of being controlled and influenced by other people or events.

Personality reflects the differences in psychological characteristics between individuals, and will not change greatly in the long-term process. Therefore, the present invention uses the Big Five personality model to construct the OCEAN personality space, and its five factors are: openness ), responsibility (conscientiousness), extraversion (extFaveFsion), agreeableness (agreeableness), neuroticism (neuroticism), expressed in vector form as P=[p _O , p _C , p _E , p _A , p _N ] ^T , The value range of the element is [-1, 1]. Among them, enlightenedness describes a person's cognitive style, seeks understanding of experience, and tolerates and explores unfamiliar situations; conscientiousness refers to the way to control, manage and regulate one's own impulses, and evaluates the individual's organization, persistence and goal-oriented behavior. Motivation; extraversion indicates the quantity and density of interpersonal interaction, the need for stimulation and the ability to obtain pleasure; agreeableness examines the individual's attitude towards other people; neuroticism reflects the individual's emotional regulation process, reflecting the individual's tendency to experience negative emotions and emotional dissatisfaction stability.

Since VGG is a class of classic deep convolutional neural network with powerful image feature extraction capabilities, this paper uses VGG-19 trained on the CK+ dataset as a video-emotional space reasoning model to establish a VGG-FACS-OCC model, as shown in Fig. 1 and Fig. 2, the Italian natural conversation video V is divided into several picture frames Image _t according to time, and the picture frame Image _t is sampled according to a fixed frequency to obtain a number of sampling frames I _i , (i=1, 2, 3, ..., n ), let the frame preprocessing function be Pre, and Pre is specifically to use the MTCNN face recognition algorithm to perform face target detection on the sampled frame I _i , and obtain the target frame set B={b ₁ , b ₂ ,...b _m }, where, b _i =(x _i , y _i , h _i , w _i , p _i ), x _i represents the abscissa of the upper left corner of the target frame, y _i represents the ordinate of the upper left corner of the target frame, h _i represents the height of the target frame, w _i represents the width of the target frame, p _i represents the confidence of the target frame; determine the height threshold h _t , width threshold w _t and confidence threshold p _t ; for any b _i ∈ B, keep B′={h _i ＞h _t and w _i ＞w _t and }; Finally, get the target frame b _* with the highest confidence p _i from B′, clip I _i according to b _* , and adjust the cropped I _i to a specific size to get Pre(I _i ), and get the emotional space vector E _i =VGG(Pre(I _i )).

Specifically, the method of converting a single frame image into an emotional space vector through VGG is as follows: Assume that the picture frame to be inferred is a pixel matrix VGG mainly involves three kinds of calculation processes: convolution layer, fully connected layer, and pooling layer. First, the formal expression of convolution operation is given:

,

Among them, s and p are the step size and the number of zero-filled layers respectively, Z^l is the input of the first layer and Z^0=I, K is the number of channels of the convolution layer, F is the height and width of the convolution kernel, L_(1+1) is the size of the input of the 1+1 layer convolutional layer, and there are: σ( ) represents a nonlinear activation function, usually a linear rectification function (ReLU) function: ReLU(x)=max(0,x), for the case of changing the number of output channels, assuming that the number of output channels is K′, generally used K′ different convolution kernels perform multiple two-dimensional convolution operations, and then connect all the results in the channel dimension.

The formal expression of the fully connected layer is as follows:

The formal expression of the pooling layer is as follows: When p → ∞, the pooling operation becomes maximum pooling, denoted as MaxPool_F, that is, the pixel value with the largest gray value in the pooling area is taken. Namely: />

The model parameters of VGG are generally determined by the pre-training process, and its forward propagation reasoning process can be formalized as: Where f_1, f_2, ..., f_n represent the functions corresponding to different levels of the neural network, /> Represents function composition, which means that the forward propagation of the model is completed through the function composition of different levels of the neural network. For one implementation of VGG, that is, VGG-19, the function expressions of each level are shown in Fig. 3 .

In order to adjust the parameters of VGG-19 so that VGG-19 has good FACS feature extraction ability and emotion classification ability, this paper uses CK+ data set to train the VGG-19 model. The CK+ dataset provides OCC emotion annotations based on FACS provided face pictures. The FACS-OCC conversion method is shown in Figure 4 below. Let the FACS-AU intensity vector be f, each dimension of which represents the intensity of FACS AU related to emotion recognition, and the value range is [0, 1]. The above-mentioned FACS-OCC conversion method is recorded as the function F2O(f), then For the picture I in the training set and the FACS-AU feature label f, the optimization objective of the VGG model is: L(VGG(I), F2O(f))=CrossEntropy(VGG(I), F2O(f)), where the cross entropy The loss is: Where n is the number of labels. For the OCC sentiment classification problem in this paper, the number of labels should be 6. The VGG model is trained with the CK+ data set by batch gradient descent, and the objective function L is minimized. With the adjustment of the model parameters, the hidden layer of VGG can extract the FACS features, and finally obtain the specific OCC emotion vector.

After obtaining the emotional space vector E _i =VGG (Pre (I _i )), calculate the mapping of the emotional space vector to the PAD mood space, with reference to the quantitative mapping relationship between the PAD mood space and emotion shown in Figure 5, wherein the emotion is surprised (surprise) There is no corresponding PAD value in the original PAD scale. We observe the PAD values corresponding to emotions similar to surprise in the original scale, and after analysis, assume that their PAD values are 0.20, 0.45, and -0.45, respectively. We assume that each emotion in the above scale is independent of each other. When one emotion has a maximum value of 1 and the rest of the emotions are all 0, it can be mapped to the corresponding PAD value on the right. Therefore, after formalizing the scale above, it is written as the following mapping relationship: f _e (e=1.0)=[m _Pe , m _Ae , m _De ] ^T , e∈{e _angry , e _disgust , e _fear , e _happy , e _sad , e _surprise }, further, we write it as a matrix multiplication form M _i =K×E _i that is convenient for computer operations, where is the transformation matrix from emotion space to mood space, and E _i is a binary vector whose modulus is 1. Through facial expression recognition technology, we get the emotion vector as E _i =[e _angry , e _disgust , e _fear , e _happy , e _sad , e _surprise ] ^T , and since only discrete correspondences are given in the table, E The value can only be 1 1, 5 0, such as [1, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0], we need to convert it into a continuous mapping function to obtain the mood vector M _i corresponding to the emotion vector E _i . Therefore, we expand the formula to obtain the mood space vector M _i =K×E′ _i , where E′ _i is the emotion vector recognized by the computer, and E′ _i can take any value within its value range. Then calculate the arithmetic mean of the mood space vector to get the average mood space vector

Finally, the average mood space vector M _v is mapped to the OCEAN personality space, and the transformation relationship from mood space to personality space is established through linear regression as:

Sophistication＝.16P+.24A+.46D

Conscientiousness＝.25P+.19D

Extraversion=.29P+.59D

Agreeableness＝.74P+.13A-.18D

Emotional stability＝.43P-.49A

Since both Sophistication and Openness are derived from Cultural, this paper assumes that Sophistication is approximately considered synonymous with Openness. Emotional stability describes the stability of individual emotions and Neuroticism describes the instability of individual emotions. Therefore, it is assumed that Emotional stability and Neuroticism are opposite to each other. Namely: Sophistication＝Openness; Emotional stability＝ -Neuroticism; thus obtain the conversion relationship from PAD mood space to OCEAN personality space:

Openness＝.16P+.24A+.46D

Conscientiousness＝.25P++.19D

Extraversion=.29P+.59D

Agreeableness＝.74P+.13A-.18D

Neuroticism＝-.43P+.49A

That is, the personality space vector P _e =Z×M _v , where B is the transformation matrix between the mood space vector and the personality space vector,

In order to verify the proposed cognitive modeling and the feasibility of its computer implementation, the present invention carried out related experiments, the research objects were 31 college students, and the experiments followed the standard procedures and paradigms of psychological experiments. The experimental process is as follows. First, a 5-10 minute interview was conducted with the test subject. The content of the interview was mainly to stimulate memories and emotions. At the same time, a video camera was used to record the subject's expression during the experiment. After the interview, the subjects filled out the traditional Big Five personality scale. Finally, the effectiveness of the model is verified by analyzing the experimental results of the scale and the results of the traditional Big Five personality scale. Relevant experimental materials, raw data and analysis results are provided in GitHub for open source access. At the same time, regarding the algorithm and result analysis and processing of the video, the operating hardware environment is as follows: Memory: 8GB; CPU: Inter(R) Core(TM) i5-7300HQ 2.50GHz 4 cores; the system operating environment is Debian 10. Taking the video data whose subject ID is "lf" as an example, the processing flow is shown in Figure 6.

The main process of the Federal Cognitive Model (EFCM) data processing is as follows. In the Big Five personality experiment, the subjects were first interviewed to obtain their video stream information, and then the video stream data was processed by using VGG-19 based on FACS-OCC emotional modeling. The emotional OCC feature data is combined with the OCC-OCEAN cognitive modeling process to finally obtain the time-series personality data of the subjects, and finally obtain the weighted personality data.

The specific processing flow is as follows. First, through the FACS-OCC emotion modeling based on CK+expression features, the six-dimensional OCC emotion time-series data of OCC is obtained from the video stream information of the subjects. The processed six-dimensional OCC emotion time-series data is shown in Figure 7 It can be seen. During the experiment, the OCC emotional activation of the subjects fluctuated to different degrees in happy (happy) and sad (sad) emotions, and they were relatively frequent, while in Angry (Angry), embarrassment (Disgust), fear ( ), and Surprise are relatively limited in activation.

According to the OCC six-dimensional emotional time-series data output above, the time-series data of PAD is obtained through the mapping process from the emotional space to the mood space, as shown in Figure 8.

Combined with the transformation theory of Mehrabian mood space to personality space, the dynamic personality recognition data can be finally obtained, that is, the small dots in Figure 9. At the same time, combined with the traditional big five personality scale in psychology, that is, the big dots in Figure 9.

According to Figure 9, it can be seen that the traditional Big Five personality data represented by the big dots falls within the credible recognition area of the algorithm. At the same time, in order to quantify the deviation of each personality, we calculated the five personality deviation rates of all the subjects to know the EFCM recognition. The accuracy of the model is known. The specific calculation method of deviation rate is explained by taking Openness as an example. Among them, Openness_rec _t represents the Openness value obtained by the algorithm for the tth frame, n represents the total number of frames, Openness represents the Openness value calculated by the personality scale, Openness_recmax represents the largest Openness value calculated by the algorithm in all frames, and Openness_recmin represents The minimum Openness value calculated by the algorithm in all frames. In order to avoid negative numbers, the absolute value of the numerator is taken.

The above formula expresses the ratio of the difference between the arithmetic mean of the Openness value obtained by the algorithm and the Openness value obtained by the personality scale, and the difference between the absolute value and the difference between the minimum value and the maximum value of Openness obtained by the algorithm, that is, the Openness deviation rate. Similarly, the deviation rates of Conscientiousness, Extraversion, Agreeableness, and Neuroticism can be obtained as follows:

According to the above calculation method, the five personality deviation rates of all subjects in the EFCM cognitive model shown in Figure 10 can be obtained as follows.

The federated cognitive model provided by this law has theoretically opened up the whole process from the input of visual information to the output of the final Big Five personality data, and combined with the formalization to conduct a detailed derivation. The model proved in the experimental stage that, except for the personality of neuroticism, which cannot be verified objectively, the experimental results show that the average deviation rate of the personality results of the four effective tests is about 20.41%, that is, the average accuracy rate is 79.56%.

For the results of neurotic personality deviation with a large deviation rate, considering that it is relatively difficult to truly capture the relatively rare negative emotions in life through recall during the actual test process, it affects the neuroticism results of the subjects in the follow-up personality comparison results and deviation rate. From the basic theory of psychology, it can be known that neuroticism is related to a large number of negative emotions. In the standard Big Five personality test experiment, the negative emotions of the subjects will not appear and cannot be deliberately stimulated. Therefore, the neurotic characteristics of the subjects in the experiment cannot be accurately measured. Test and record, objectively cause abnormal results of Neuroticism personality. If the model is applied to the actual scene, it will be recorded and analyzed unconsciously by the subjects, theoretically it can effectively test the neurotic personality. Therefore, the next step of the research is to observe and experiment with massive data in massive scenarios with the permission of the subjects to verify the effectiveness of the federated cognitive model under large-scale observations.

The preferred embodiments of the present invention have been described above; it should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the equipment and structures that are not described in detail should be understood as being implemented in a common manner in the art; Any person skilled in the art can make many possible changes and modifications without departing from the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which will not affect the essence of the present invention; therefore, anyone who does not depart from the present invention For the content of the technical solution of the invention, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention still belong to the protection scope of the technical solution of the present invention.

Claims (1)

1. the OCC-PAD-OCEAN federal cognitive modeling method based on emotion, it is characterized in that, the method comprises the following steps: S1 constructs the VGG-FACS-OCC model, sets the FACS-AU intensity vector as f, sets the value range as [0,1], and marks the conversion of FACS-OCC as a function F2O(f); The optimization objective of the VGG model is set as: L(VGG(I),F2O(f))＝CrossEntropy(VGG(I),F2O(f)) The cross-entropy loss function is set to: The VGG model is trained with the CK+ data set through batch gradient descent to minimize the objective function L; Divide the video under test into several picture frames Image _t according to time, and sample the picture frame Image _t at a fixed frequency to obtain a number of sampling frames I _i , (i=1,2,3,...,n); Use the MTCNN face recognition algorithm to perform face target detection on the sampled frame I _i , and obtain the target frame set B={b ₁ , b ₂ ,...b _m }, where b _i =(xi _, y _i , h _i , w _i , p _i ), x _i represents the abscissa of the upper left corner of the target frame, y _i represents the vertical coordinate of the upper left corner of the target frame, h i represents the height of the target frame, _{w i represents} the width of the target frame, and p _i represents the target frame confidence level; Determine height threshold h _t , width threshold w _t and confidence threshold p _t ; for any b _i ∈ B, keep B′={h _i >h _t and w _i >w _t and S123 Obtain the target frame b _* with the highest confidence p _i from B′, clip I _i according to b _* , and adjust the cropped I _i to a specific size to obtain Pre(I _i ); Perform FACS feature extraction based on the hidden layer of VGG, map the preprocessed sampling frame I _i to the OCC emotion space, and obtain the emotion space vector E _i , the emotion space vector E _i =VGG(Pre(I _i )); S2 calculates the mood space vector M _i =K×E _i , wherein K is the conversion matrix of the parameter quantization mapping relationship between the OCC emotion space and the PAD mood space; Continuing the mapping relationship between mood space and personality space, we get M _i =K×E′ _i ; Compute the mean mood space vector S3 maps the average mood space vector to the OCEAN personality space, and obtains the personality space vector through the mood space vector; the personality space vector P _e =Z×M _v , where Z is the conversion matrix from the PAD mood space to the OCEAN personality space.