WO2021217973A1 - Emotion information recognition method and apparatus, and storage medium and computer device - Google Patents

Abstract

Disclosed are an emotion information recognition method and apparatus, and a storage medium and a computer device, which relate to the technical field of artificial intelligence, and aim to respectively process a human body posture and facial expression, which have been obtained, into a posture matrix and emotion vector information, and to simultaneously process, by means of a pre-trained emotion intensity recognition model, the posture matrix and the vector information to obtain emotion intensity data and correspondingly recognize an emotion type, thereby improving the accuracy and efficiency of emotion information recognition. The method comprises: receiving an emotion information recognition request, wherein the emotion information recognition request carries human body posture information; transforming the human body posture information into a posture matrix including posture feature points by using a preset posture transformation algorithm; processing the posture matrix according to a preset emotion intensity algorithm, so as to obtain emotion intensity data; and retrieving and feeding back a corresponding emotion type according to the emotion intensity data. In addition, the present application further relates to blockchain technology, and the emotion intensity data may be stored in a blockchain.

Description

Emotional information recognition method, device, storage medium and computer equipment

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 28, 2020, the application number is 202010349534.0, and the invention title is "emotional information recognition method, device, storage medium and computer equipment", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the field of artificial intelligence technology, in particular to an emotional information recognition method, device, storage medium, and computer equipment.

Background technique

With the development of big data, allowing robots to have social and service capabilities and the ability to read human emotional intensity and fluctuations in real time during human-computer interaction has increasingly become people's desires and needs. In the actual business processing process, if the robot can detect human emotion fluctuations in time and can make adaptive adjustments according to human response, human dissatisfaction will be alleviated, and the robot's performance will be more accepted by users.

The inventor realizes that at present, the traditional emotion information recognition technology is only to infer people’s emotions through facial expressions. However, this emotion information recognition method ignores the emotional intensity of the body language that people naturally make in social situations. The role played has led to low accuracy of emotion information recognition and low efficiency of emotion information recognition.

Summary of the invention

In view of this, the present application provides an emotional information recognition method, device, storage medium, and computer equipment, the main purpose of which is to improve the accuracy and efficiency of emotional information recognition.

According to one aspect of this application, a method for identifying emotional information is provided, including:

Receiving an emotional information recognition request, where the emotional information recognition request carries human posture information;

Using a preset posture conversion algorithm to convert the human posture information into a posture matrix containing posture feature points;

Processing the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

According to the emotion intensity data, the corresponding emotion type is retrieved and fed back.

According to the second aspect of the present application, there is provided an emotional information recognition device, including:

A receiving unit, configured to receive an emotional information recognition request, where the emotional information recognition request carries human posture information;

A conversion unit, configured to convert the human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm;

A processing unit, configured to process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

The feedback unit is used to retrieve and feed back the corresponding emotion type according to the emotion intensity data.

According to the third aspect of the present application, there is provided a storage medium in which at least one executable instruction is stored, and the executable instruction causes a processor to perform the following steps: receiving an emotional information recognition request, and the emotional information recognizing The request carries human posture information; the human posture information is converted into a posture matrix containing posture feature points using a preset posture conversion algorithm; the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data ; According to the emotional intensity data, retrieve and feed back the corresponding emotional type.

According to the fourth aspect of the present application, a computer device is provided, which includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus, and The memory is used to store at least one executable instruction, the executable instruction causes the processor to perform the following steps: receiving an emotional information recognition request, the emotional information recognition request carries human posture information; using a preset posture conversion The algorithm converts the human body posture information into a posture matrix containing posture feature points; processes the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, retrieves and feeds back the corresponding emotion type.

This application can improve the accuracy and efficiency of emotional information recognition.

Description of the drawings

By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only used for the purpose of illustrating the preferred embodiments, and are not considered as a limitation to the application. Also, throughout the drawings, the same reference symbols are used to denote the same components. In the attached picture:

Fig. 1 shows a flow chart of an emotional information recognition method provided by an embodiment of the present application;

FIG. 2 shows a schematic diagram of human body feature points based on Euler angles according to an embodiment of the present application;

FIG. 3 shows a schematic structural diagram of an emotional information recognition device provided by an embodiment of the present application;

Fig. 4 shows a schematic diagram of the physical structure of a computer device provided by an embodiment of the present application.

Detailed ways

Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the drawings show exemplary embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

The technical solution of this application can be applied to the fields of artificial intelligence, block chain and/or big data technology, and the data involved can be stored in a database or can be stored through a block chain, which is not limited by this application.

As mentioned in the background art, at present, the traditional emotion information recognition technology only stays at inferring human emotions through facial expressions. However, this emotion information recognition method ignores the emotional intensity of the body language that people naturally make in social situations. As a result, the accuracy of emotion information recognition is not high, and the efficiency of emotion information recognition is relatively low.

In order to solve the foregoing problem, an embodiment of the present application provides a method for identifying emotional information. As shown in FIG. 1, the method includes:

101. Receive an emotional information recognition request, where the emotional information recognition request carries human posture information.

Wherein, the emotional information recognition request may specifically be sent by the server. In actual application scenarios, images or images with human body posture information can be acquired through a camera placed inside the robot, and the human body posture information can be used for emotion analysis, so that the robot can take different types of emotions to the user according to the obtained emotion types. Measures, such as handling business in advance, or changing windows, etc., to improve business handling efficiency.

102. Use a preset posture conversion algorithm to convert the human posture information into a posture matrix containing posture feature points.

Wherein, the posture conversion algorithm may specifically express the rotation degrees of freedom of each joint point of the human body through Euler angles. For the obtained human body posture information, it can be processed by a preset algorithm to obtain a matrix containing 13 feature points. Since the human body joints have angular rotation motion, the embodiment of this application uses Euler angles to determine the position of each feature point. The rotation degree of freedom is expressed, so that the coordinate position and the rotation degree of freedom of each feature point of the human body in each frame of image can be obtained through the human body posture information. Through step 102, the posture of the human body can be abstracted into a posture matrix represented by 13 feature points, and the matrix can be subsequently analyzed to obtain emotion types corresponding to different posture matrices.

It should be noted that the number of feature points corresponding to the embodiment of the present application can be set according to the requirements of the service type. For example, if the accuracy requirement is high, the finger joints can also be abstracted as feature points.

103. Process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data.

Wherein, the emotion intensity algorithm may specifically include processing the posture matrix through a pre-trained emotion intensity model. For the embodiment of the present application, the emotion intensity model may specifically be a two-layer LSTM-RNN structure. As the prior art only uses facial recognition as a dimension to recognize emotional information, it is easy to cause recognition errors and result in low recognition accuracy. However, the embodiment of this application is innovative on this basis and adopts a two-layer LSTM-RNN structure, which can Combining the two dimensions of facial recognition and human posture to recognize emotion types greatly improves the accuracy of emotion information recognition. Specifically, by processing the posture matrix according to a preset emotion intensity algorithm, corresponding emotion intensity data can be obtained, and the emotion intensity data can be used to correspondingly search for emotion types, so as to adopt a corresponding processing method.

104. According to the emotion intensity data, retrieve and feed back the corresponding emotion type.

Wherein, after the emotion intensity data is obtained, the corresponding emotion type is retrieved locally, and the emotion type is used to respond to the emotion information recognition request. For example, if the emotion intensity is 1, the emotion type corresponding to the emotion intensity 1 can be found locally as anger. After the emotion type of anger is fed back, the robot can be controlled to take measures to handle the business in advance for the user. For the embodiment of the present application, the corresponding relationship between the emotion intensity data and the emotion type can be established in advance, and the emotion intensity data, the emotion type, and the corresponding relationship between the emotion intensity data and the emotion type can be saved At the local level, different measures can be taken to appease. In practical application scenarios, such as when handling business at the front desk of a bank, users often wait and queue. During this period, users may be impatient, confused, and need help in self-service handling. In the process of estimating the emotional strength of the human body posture, it can comfort the user and assist the user in handling business according to the user's expression and action posture.

Further, in order to better explain the process of the foregoing emotion information recognition method, as a refinement and extension of the foregoing embodiment, the embodiments of the present application provide several optional embodiments, but are not limited thereto, and the details are as follows:

In an optional embodiment of the present application, the step 103 may specifically include: using a pre-trained emotion intensity model to process the posture matrix and the obtained emotion vector information to obtain the emotion intensity.

Wherein, the pre-trained emotional intensity model may be an LSTM-RNN structure, which can capture millimeter-level differential streams, so this structure is very suitable for handling complex, variable length, and highly intrinsically related human postures sequence. For the embodiment of the present application, a two-layer LSTM-RNN structure can be adopted, and the structure input variable can be a variable-length action series {X ₁ , X ₂ , X ₃ ,...X _n-1 , X _n }, the variable-length action sequence may specifically be a human body posture image, and the X _n may be any frame of the human body posture image in the human body posture image. In addition, for the embodiment of the present application, the emotion vector corresponding to the facial expression can also be input as another parameter. The emotion vector can be used to represent the emotion type of the actual person, specifically, each human face in the human body posture image An expression can correspond to a different emotion type. The emotion type can be represented by a value between 0 and 1, for example, 1 is used to represent anger, 2 is used to represent surprise, etc., and the human body posture image and the emotion vector At the same time, it is input into the pre-trained emotional intensity model for processing, and emotional intensity data can be obtained. For example, for the human posture of slightly open arms, if the recognized facial expression is normal, then the emotional strength of calmness can be obtained; but if the recognized facial expression is staring, then the emotional strength of surprise can be obtained. That is to say, by combining the emotion type with the human body posture, the accuracy of recognition can be improved and correct feedback can be made. For example, if a person’s emotions are dissatisfaction, he may frown, and if irritated by improper behavior, he may express strong dissatisfaction by shrugging his shoulders. But if you can detect this emotional fluctuation in time and make adaptive adjustments based on people's reactions, people's dissatisfaction will be alleviated and users will be more accepted. For another example, the movement of slightly opening the arms is just a normal posture when a person expresses his thoughts calmly; but if his expression is surprised, it will be much more surprised than the naturally hanging arms. Therefore, in order to enable the robot to obtain a more comprehensive analysis of human body posture actions, the emotion type can be included in the process of emotion intensity estimation.

For the embodiment of the present application, using the pre-trained emotion strength model to process the posture matrix and the obtained emotion vector information to obtain the emotion strength may specifically include: using a sigmoid function to simultaneously perform the input posture matrix and emotion vector information. Process and output the obtained emotional intensity data, which is stored in the blockchain.

Specifically, the posture matrix extracted from the human posture image and the emotion vector information are simultaneously input into the pre-trained emotion intensity model, and the sigmoid function in the model can be called for processing. The sigmoid function can be used to convert any real number Converted to a certain number between 0-1 as the probability. For example, after the sigmoid function processes the posture matrix and emotion vector information, the probability of different emotion intensity data can be obtained, such as anger 93%, less than 5% , Happy 1%, excited 1%, the emotional intensity data with the highest probability can be output.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned emotional intensity data, the above-mentioned emotional intensity data may also be stored in a node of a blockchain.

Wherein, in order to ensure the privacy and security of the emotional intensity data, the emotional intensity data may also be stored in a node of the blockchain. Specifically, a blockchain network can be established in advance, and the recording nodes in the blockchain network can be used to record emotional intensity data, the emotional intensity data can be packaged and stored in a new block, and the generated management key Save it in the record node for retrieval and feedback when needed. In the embodiment of the present application, the emotional intensity data is stored through the blockchain technology, which can greatly ensure the security of the emotional intensity data, and it is easy to retrieve the data, and can improve the efficiency of emotion recognition.

In another optional embodiment of the present application, the step 102 may specifically include: obtaining Euler angle parameters of each feature point; and determining the posture of each feature point based on the human body static model coordinate system according to the Euler angle parameters matrix.

Among them, due to the difference in the length of the human body skeleton, the corresponding posture representations of different human bodies are different. Therefore, in order to simplify the complexity of the data, the feature of the skeleton length is eliminated, and only the rotation freedom of the joint points is retained. Euler angles indicate that the Euler angles can be used to determine a set of three independent angular parameters of the position of a fixed-point rotating rigid body. For example, the rotational freedom of a joint point can be represented by a set of three angular parameters of rpy.

A standardization of the body feature data of different people is realized by inputting a pre-defined human skeleton model. Each feature point in the pre-defined model has a pre-defined coordinate system. The key point extraction technology in OpenPose can be used to determine that each joint is in the human body. At the position above, the OpenPose human body gesture recognition is an open source library developed by Carnegie Mellon University (CMU) based on convolutional neural networks and supervised learning and using caffe as the framework. It can realize posture estimation of human body movements, facial expressions, and finger movements. It is suitable for single and multiple people, and has excellent robustness. Specifically, the input is an image, the basic model can be VGG19, and the model output is a matrix that can be represented by the above posture. For the embodiment of this application, only the key point information needs to be extracted as the model input to send it to the two-layer LSTM-RNN Network. At the same time, OpenCV also provides a call interface for the openpose open source framework, and key point information can also be calculated in this way. Next, extract the point cloud near the extracted key points, estimate the orientation of the key points relative to the same key point of the predefined model, and express the transformation relationship of each joint relative to the predefined joint point by Euler angles. . In this way, the posture of the person at time i (frame) can be represented by the following matrix X _i :

Among them, r, p, y in the matrix can respectively represent the Euler angle parameters of each feature point.

In yet another optional embodiment of the present application, the method may further include: performing recognition processing on the acquired facial expression information by using a preset facial recognition algorithm to obtain corresponding emotion vector information.

Wherein, the specific process of the facial recognition algorithm may include: reading facial expression images, using the top of the head as a reference point to estimate the approximate position of facial features, and evenly setting mark points on the contours of each feature part of the face; passing through the center of the brow and the pupils The central axis fitted by the midpoint of the connection line and the three points at the center of the mouth divides the face into two symmetrical parts. Without scaling, translation, or rotation, the image will be adjusted to be symmetrical with respect to the central axis. Adjust the mark points of the to the same horizontal line, and establish a facial expression shape model; in the facial expression shape model, divide the left eye/right eye, left eyebrow/right eyebrow and mouth into different areas, and define these areas as feature candidate areas; For each feature candidate area, the feature vector is extracted by the difference image method. By performing the difference operation between all the image sequences in the image processed in the previous step and the neutral expression image in the database, the average value of the difference value in each feature candidate area is the largest The facial expression feature vector is extracted from the image sequence.

After the facial expression feature vector is obtained, the emotion vector data corresponding to the facial expression feature vector is retrieved locally. The emotion vector data may indicate the type of emotion expressed by facial expressions. For example, through the facial expression of frowning, the emotion vector dissatisfaction can be correspondingly obtained.

In still another optional embodiment of the present application, the method may further include: according to the human body posture information, establishing a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system; The coordinates of the nodes are related, and the joint points are determined as the characteristic points of the posture of the human body.

Wherein, as shown in Figure 2, the G coordinate system can be a human body static model coordinate system, and the representation method of the coordinate system can be specifically described as follows: "Skeleton tree", the relative relationship between feature points and feature points in the skeleton tree is statically stored as a predefined model; secondly, since everyone's body structure is the same but the skeleton length is different, homogeneous transformation can be introduced The matrix T represents the rigid transformation of different individuals with respect to the corresponding points on the static model. The position of any point can be obtained by matrix multiplication.

In still another optional embodiment of the present application, the method may further include: training an emotional strength model according to the RNN-LSTM model, sample pose data, and preset emotional pose tags.

Specifically, before model training, pre-training needs to be performed by making a data set or using a data set on the network. For example, for a certain posture, the actual emotion of the observed person needs to be determined and data labeling is completed. The specific process of the model training may include: RNN-based improved long-term and short-term memory modeling. Since many data in real life have both temporal and spatial features, such as human body motion trajectory, continuous frames of video, etc., human body The same is true for posture, each time point corresponds to a set of data, and the data often has certain spatial characteristics. Therefore, in order to carry out classification and prediction work on such a time series, it is necessary to model and extract features in time and space. The commonly used time modeling tool is the Recurrent Neural Network (RNN) correlation model (LSTM). Because of its unique gate structure design and powerful extraction of time series features, it is widely used in forecasting problems and has achieved good results. . The traditional LSTM structure includes three structures: input gate, output gate, forget gate, and a neural node (cell), where the input can be the body's posture representation in the current frame at time t, The output can be a posture descriptor, which is used to describe the type of current posture. For the embodiment of the present application, a double-layer LSTM-RNN structure is formed by connecting n LSTM structures horizontally, because a continuous image stream is often required when determining the human body posture, so through {X ₁ , X ₂ , X ₃ ,. ..... X _n-1 ,X _n } to represent such a video stream and serve as the input of the model.

It should be noted that the training of the model requires a data set for pre-training, and then a second training is performed by creating a data set to achieve a relatively robust effect. For example, the current posture image stream of a person is recorded through the camera, the posture description matrix is obtained through the key point extraction method of OpenPose, and the current person's emotion type strength is asked to complete the data labeling, and then the training is carried out. For the embodiment of the present application, the structure of the LSTM unit is still adopted, but the structure of a double-layer LSTM and a fully connected layer are adopted. The double-layer structure can increase the correlation detection of the timing.

This application provides an emotional information recognition method. Compared with the prior art inferring human emotions only through facial expressions, this application receives an emotional information recognition request that carries human posture information; using predictions The posture conversion algorithm is assumed to convert the human posture information into a posture matrix containing posture feature points; the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, retrieve and merge Feedback corresponding emotion type. Therefore, the accuracy and efficiency of emotion information recognition can be improved through the dual dimensions of human body posture and facial expression. In addition, this application also uses blockchain technology to store data, which can improve the security of emotional information.

Further, as a specific implementation of FIG. 1, an embodiment of the present application provides an emotional information recognition device. As shown in FIG. 3, the device includes: a receiving unit 21, a conversion unit 22, a processing unit 23 and a feedback unit 24.

The receiving unit 21 may be configured to receive an emotion information recognition request, where the emotion information recognition request carries human posture information;

The conversion unit 22 may be used to convert the human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm;

The processing unit 23 may be configured to process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

The feedback unit 24 may be used to retrieve and feed back the corresponding emotion type according to the emotion intensity data.

Further, the processing unit 23 includes:

The processing module 231 may be used to process the posture matrix and the obtained emotion vector information using a pre-trained emotion intensity model to obtain the emotion intensity.

Further, the processing module 231 may be specifically configured to simultaneously process the input posture matrix and emotion vector information by using the sigmoid function, and output the obtained emotion intensity data.

Further, the processing module 231 may be specifically used to store the emotional intensity data using blockchain technology.

Further, the conversion unit 22 includes:

The obtaining module 221 may be used to obtain Euler angle parameters of each feature point;

The determining module 222 may be used to determine the posture matrix of each feature point in the human body static model coordinate system according to the Euler angle parameters.

Further, the device further includes:

The recognition unit 25 may be used to perform recognition processing on the acquired facial expression information by using a preset facial recognition algorithm to obtain corresponding emotion vector information.

Further, the device further includes:

The establishing unit 26 may be used to establish, according to the human body posture information, a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system;

The determining unit 27 may be used to determine the coordinates of each joint point by matrix multiplication, and determine the joint point as a feature point of the posture of the human body.

Further, the device further includes:

The training unit 28 may be used to train the emotional intensity model according to the RNN-LSTM model, sample pose data, and preset emotional pose labels.

Based on the above method shown in FIG. 1, correspondingly, an embodiment of the present application further provides a storage medium in which at least one executable instruction is stored, and the executable instruction causes the processor to perform the following steps: receiving Emotion information recognition request, said emotion information recognition request carries human body posture information; using a preset posture conversion algorithm to convert the human body posture information into a posture matrix containing posture feature points; according to the preset emotion intensity algorithm The posture matrix is processed to obtain emotion intensity data; according to the emotion intensity data, the corresponding emotion type is retrieved and fed back. Optionally, the executable instruction can also implement other steps of the method in the foregoing embodiment when executed by the processor, which will not be repeated here. Further optionally, the storage medium involved in the present application may be a computer-readable storage medium, and the storage medium, such as a computer-readable storage medium, may be non-volatile or volatile.

Based on the above-mentioned method shown in FIG. 1 and the embodiment of the apparatus shown in FIG. 3, an embodiment of the present application also provides a computer device. As shown in FIG. 4, a processor 31 and a communication interface (Communications Interface) 32. A memory (memory) 33, and a communication bus 34. Among them, the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34. The communication interface 34 is used to communicate with other devices, such as network elements such as user terminals or other servers. The processor 31 is configured to execute a program, and specifically can execute relevant steps in the foregoing embodiment of the emotion information recognition method. Specifically, the program may include program code, and the program code includes computer operation instructions. The processor 31 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs. The memory 33 is used to store programs. The memory 33 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), for example, at least one disk memory. The program can specifically be used to make the processor 31 perform the following operations: receive an emotion information recognition request, the emotion information recognition request carries human posture information; use a preset posture conversion algorithm to convert the human posture information to include posture features Point posture matrix; process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data; retrieve and feed back the corresponding emotion type according to the emotion intensity data.

Through the technical solution of the present application, it is possible to receive an emotion information recognition request, the emotion information recognition request carries human body posture information; use a preset posture conversion algorithm to convert the human body posture information into a posture matrix containing posture feature points; The posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, the corresponding emotion type is retrieved and fed back. Therefore, the accuracy and efficiency of emotion information recognition can be improved through the dual dimensions of human body posture and facial expression. In addition, this application also uses blockchain technology to store data, which can improve the security of emotional information.

The blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

It should be noted that, for other corresponding descriptions of the functional modules involved in the emotional information recognition device provided in the embodiment of the present application, reference may be made to the corresponding description of the method shown in FIG. 1, and details are not repeated here.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

It can be understood that the relevant features in the above method and device can be referred to each other. In addition, the “first”, “second”, etc. in the foregoing embodiments are used to distinguish the embodiments, and do not represent the advantages and disadvantages of the embodiments.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

The algorithms and displays provided here are not inherently related to any particular computer, virtual system or other equipment. Various general-purpose systems can also be used with the teaching based on this. Based on the above description, the structure required to construct this type of system is obvious. In addition, this application is not aimed at any specific programming language. It should be understood that various programming languages can be used to implement the content of the application described herein, and the above description of a specific language is for the purpose of disclosing the best embodiment of the application.

In the instructions provided here, a lot of specific details are explained. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and technologies are not shown in detail, so as not to obscure the understanding of this specification.

Similarly, it should be understood that, in order to simplify the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, Figure, or its description. However, the disclosed method should not be interpreted as reflecting the intention that the claimed application requires more features than the features explicitly recorded in each claim. More precisely, as reflected in the following claims, the inventive aspect lies in less than all the features of a single embodiment disclosed previously. Therefore, the claims following the specific embodiment are thus explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the application.

Those skilled in the art can understand that it is possible to adaptively change the modules in the device in the embodiment and set them in one or more devices different from the embodiment. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and/or processes or units are mutually exclusive, any combination can be used to compare all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or methods disclosed in this manner or All the processes or units of the equipment are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means that they are within the scope of the present application. Within and form different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present application may be implemented by hardware, or by software modules running on one or more processors, or by a combination of them. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present application. This application can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present application may be stored on a computer-readable medium, or may have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of multiple such elements. The application can be realized by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied in the same hardware item. The use of the words first, second, and third, etc. do not indicate any order. These words can be interpreted as names.

Claims (20)

1. An emotional information recognition method, which includes:

   Receiving an emotional information recognition request, where the emotional information recognition request carries human posture information;

   Using a preset posture conversion algorithm to convert the human posture information into a posture matrix containing posture feature points;

   Processing the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

   According to the emotion intensity data, the corresponding emotion type is retrieved and fed back.
2. The method according to claim 1, wherein the processing the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data comprises:

   The pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data.
3. The method according to claim 2, wherein said using a pre-trained emotion intensity model to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data comprises:

   The sigmoid function is used to process the input posture matrix and emotion vector information at the same time, and the obtained emotion intensity data is output, and the emotion intensity data is stored in the blockchain.
4. The method according to claim 3, wherein said using a preset posture conversion algorithm to convert said human posture information into a posture matrix containing posture feature points comprises:

   Obtain the Euler angle parameters of each feature point;

   The posture matrix of each feature point in the coordinate system of the human body static model is determined according to the Euler angle parameters.
5. The method according to claim 4, wherein the method further comprises: before the posture matrix and the obtained emotion vector information are processed by using a pre-trained emotion intensity model to obtain emotion intensity data, the method further comprises:

   A preset facial recognition algorithm is used to recognize and process the acquired facial expression information to obtain the corresponding emotion vector information.
6. The method according to claim 5, wherein before said converting said human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm, said method further comprises:

   According to the human body posture information, establish a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system;

   The coordinates of each joint point are determined by matrix multiplication, and the joint points are determined as the characteristic points of the posture of the human body.
7. 7. The method according to claim 6, wherein said using a pre-trained emotional intensity model to process said posture matrix and the acquired emotional appropriate amount of information, and before obtaining emotional intensity data, said method further comprises:

   According to the RNN-LSTM model, sample pose data, and preset emotional pose labels, the emotional intensity model is trained.
8. An emotional information recognition device, which includes:

   A receiving unit, configured to receive an emotional information recognition request, where the emotional information recognition request carries human posture information;

   A conversion unit, configured to convert the human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm;

   A processing unit, configured to process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

   The feedback unit is used to retrieve and feed back the corresponding emotion type according to the emotion intensity data.
9. A storage medium on which a computer program is stored, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform the following steps:

   Receiving an emotional information recognition request, where the emotional information recognition request carries human posture information;

   Using a preset posture conversion algorithm to convert the human posture information into a posture matrix containing posture feature points;

   Processing the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

   According to the emotion intensity data, the corresponding emotion type is retrieved and fed back.
10. The storage medium according to claim 9, wherein when the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data, the following steps are specifically executed:

    The pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data.
11. 10. The storage medium according to claim 10, wherein the pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data, specifically performing the following steps:

    The sigmoid function is used to process the input posture matrix and emotion vector information at the same time, and the obtained emotion intensity data is output, and the emotion intensity data is stored in the blockchain.
12. The storage medium according to claim 11, wherein the following steps are specifically performed when the human body posture information is converted into a posture matrix containing posture feature points by using a preset posture conversion algorithm:

    Obtain the Euler angle parameters of each feature point;

    The posture matrix of each feature point in the coordinate system of the human body static model is determined according to the Euler angle parameters.
13. The storage medium according to claim 12, wherein the pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information, and before the emotion intensity data is obtained, the executable instruction further causes the processor implement:

    A preset facial recognition algorithm is used to recognize and process the acquired facial expression information to obtain the corresponding emotion vector information.
14. The storage medium according to claim 13, wherein, before the human body posture information is converted into a posture matrix containing posture feature points by using a preset posture conversion algorithm, the executable instruction further causes the processor to execute:

    According to the human body posture information, establish a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system;

    The coordinates of each joint point are determined by matrix multiplication, and the joint points are determined as the characteristic points of the posture of the human body.
15. A computer device includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus, and the memory is used to store at least one executable Instructions, the executable instructions cause the processor to perform the following steps:

    Receiving an emotional information recognition request, where the emotional information recognition request carries human posture information;

    Using a preset posture conversion algorithm to convert the human posture information into a posture matrix containing posture feature points;

    Processing the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;

    According to the emotion intensity data, the corresponding emotion type is retrieved and fed back.
16. 15. The computer device according to claim 15, wherein when the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data, the following steps are specifically executed:

    The pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data.
17. 15. The computer device according to claim 16, wherein the pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information to obtain emotion intensity data, specifically performing the following steps:

    The sigmoid function is used to process the input posture matrix and emotion vector information at the same time, and the obtained emotion intensity data is output, and the emotion intensity data is stored in the blockchain.
18. 18. The computer device according to claim 17, wherein the following steps are specifically performed when the human body posture information is converted into a posture matrix containing posture feature points by using a preset posture conversion algorithm:

    Obtain the Euler angle parameters of each feature point;

    The posture matrix of each feature point in the coordinate system of the human body static model is determined according to the Euler angle parameters.
19. 18. The computer device according to claim 18, wherein the pre-trained emotion intensity model is used to process the posture matrix and the obtained emotion vector information, and before the emotion intensity data is obtained, the executable instruction further causes the processor implement:

    The preset facial recognition algorithm is used to recognize the acquired facial expression information to obtain the corresponding emotion vector information.
20. 20. The computer device according to claim 19, wherein, before the human body posture information is converted into a posture matrix containing posture feature points by using a preset posture conversion algorithm, the executable instruction further causes the processor to execute:

    According to the human body posture information, establish a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system;

    The coordinates of each joint point are determined by matrix multiplication, and the joint points are determined as the characteristic points of the posture of the human body.

Images