WO2021217973A1 - Procédé et appareil de reconnaissance d'informations d'émotion, support de stockage et dispositif informatique - Google Patents

Procédé et appareil de reconnaissance d'informations d'émotion, support de stockage et dispositif informatique Download PDF

Info

Publication number
WO2021217973A1
WO2021217973A1 PCT/CN2020/111036 CN2020111036W WO2021217973A1 WO 2021217973 A1 WO2021217973 A1 WO 2021217973A1 CN 2020111036 W CN2020111036 W CN 2020111036W WO 2021217973 A1 WO2021217973 A1 WO 2021217973A1
Authority
WO
WIPO (PCT)
Prior art keywords
posture
emotion
information
matrix
emotion intensity
Prior art date
Application number
PCT/CN2020/111036
Other languages
English (en)
Chinese (zh)
Inventor
喻凌威
周宸
周宝
陈远旭
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021217973A1 publication Critical patent/WO2021217973A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • This application relates to the field of artificial intelligence technology, in particular to an emotional information recognition method, device, storage medium, and computer equipment.
  • the inventor realizes that at present, the traditional emotion information recognition technology is only to infer people’s emotions through facial expressions.
  • this emotion information recognition method ignores the emotional intensity of the body language that people naturally make in social situations.
  • the role played has led to low accuracy of emotion information recognition and low efficiency of emotion information recognition.
  • the present application provides an emotional information recognition method, device, storage medium, and computer equipment, the main purpose of which is to improve the accuracy and efficiency of emotional information recognition.
  • a method for identifying emotional information including:
  • the corresponding emotion type is retrieved and fed back.
  • an emotional information recognition device including:
  • a receiving unit configured to receive an emotional information recognition request, where the emotional information recognition request carries human posture information
  • a conversion unit configured to convert the human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm
  • a processing unit configured to process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data
  • the feedback unit is used to retrieve and feed back the corresponding emotion type according to the emotion intensity data.
  • a storage medium in which at least one executable instruction is stored, and the executable instruction causes a processor to perform the following steps: receiving an emotional information recognition request, and the emotional information recognizing The request carries human posture information; the human posture information is converted into a posture matrix containing posture feature points using a preset posture conversion algorithm; the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data ; According to the emotional intensity data, retrieve and feed back the corresponding emotional type.
  • a computer device which includes a processor, a memory, a communication interface, and a communication bus.
  • the processor, the memory, and the communication interface communicate with each other through the communication bus, and
  • the memory is used to store at least one executable instruction, the executable instruction causes the processor to perform the following steps: receiving an emotional information recognition request, the emotional information recognition request carries human posture information; using a preset posture conversion
  • the algorithm converts the human body posture information into a posture matrix containing posture feature points; processes the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, retrieves and feeds back the corresponding emotion type.
  • This application can improve the accuracy and efficiency of emotional information recognition.
  • Fig. 1 shows a flow chart of an emotional information recognition method provided by an embodiment of the present application
  • FIG. 2 shows a schematic diagram of human body feature points based on Euler angles according to an embodiment of the present application
  • FIG. 3 shows a schematic structural diagram of an emotional information recognition device provided by an embodiment of the present application
  • Fig. 4 shows a schematic diagram of the physical structure of a computer device provided by an embodiment of the present application.
  • the technical solution of this application can be applied to the fields of artificial intelligence, block chain and/or big data technology, and the data involved can be stored in a database or can be stored through a block chain, which is not limited by this application.
  • the traditional emotion information recognition technology only stays at inferring human emotions through facial expressions.
  • this emotion information recognition method ignores the emotional intensity of the body language that people naturally make in social situations. As a result, the accuracy of emotion information recognition is not high, and the efficiency of emotion information recognition is relatively low.
  • an embodiment of the present application provides a method for identifying emotional information. As shown in FIG. 1, the method includes:
  • the emotional information recognition request may specifically be sent by the server.
  • images or images with human body posture information can be acquired through a camera placed inside the robot, and the human body posture information can be used for emotion analysis, so that the robot can take different types of emotions to the user according to the obtained emotion types. Measures, such as handling business in advance, or changing windows, etc., to improve business handling efficiency.
  • the posture conversion algorithm may specifically express the rotation degrees of freedom of each joint point of the human body through Euler angles.
  • the obtained human body posture information it can be processed by a preset algorithm to obtain a matrix containing 13 feature points. Since the human body joints have angular rotation motion, the embodiment of this application uses Euler angles to determine the position of each feature point.
  • the rotation degree of freedom is expressed, so that the coordinate position and the rotation degree of freedom of each feature point of the human body in each frame of image can be obtained through the human body posture information.
  • the posture of the human body can be abstracted into a posture matrix represented by 13 feature points, and the matrix can be subsequently analyzed to obtain emotion types corresponding to different posture matrices.
  • the number of feature points corresponding to the embodiment of the present application can be set according to the requirements of the service type. For example, if the accuracy requirement is high, the finger joints can also be abstracted as feature points.
  • the emotion intensity algorithm may specifically include processing the posture matrix through a pre-trained emotion intensity model.
  • the emotion intensity model may specifically be a two-layer LSTM-RNN structure.
  • the prior art only uses facial recognition as a dimension to recognize emotional information, it is easy to cause recognition errors and result in low recognition accuracy.
  • the embodiment of this application is innovative on this basis and adopts a two-layer LSTM-RNN structure, which can Combining the two dimensions of facial recognition and human posture to recognize emotion types greatly improves the accuracy of emotion information recognition.
  • corresponding emotion intensity data can be obtained, and the emotion intensity data can be used to correspondingly search for emotion types, so as to adopt a corresponding processing method.
  • the emotion intensity data retrieve and feed back the corresponding emotion type.
  • the corresponding emotion type is retrieved locally, and the emotion type is used to respond to the emotion information recognition request. For example, if the emotion intensity is 1, the emotion type corresponding to the emotion intensity 1 can be found locally as anger. After the emotion type of anger is fed back, the robot can be controlled to take measures to handle the business in advance for the user.
  • the corresponding relationship between the emotion intensity data and the emotion type can be established in advance, and the emotion intensity data, the emotion type, and the corresponding relationship between the emotion intensity data and the emotion type can be saved At the local level, different measures can be taken to appease. In practical application scenarios, such as when handling business at the front desk of a bank, users often wait and queue. During this period, users may be impatient, confused, and need help in self-service handling. In the process of estimating the emotional strength of the human body posture, it can comfort the user and assist the user in handling business according to the user's expression and action posture.
  • the step 103 may specifically include: using a pre-trained emotion intensity model to process the posture matrix and the obtained emotion vector information to obtain the emotion intensity.
  • the pre-trained emotional intensity model may be an LSTM-RNN structure, which can capture millimeter-level differential streams, so this structure is very suitable for handling complex, variable length, and highly intrinsically related human postures sequence.
  • LSTM-RNN structure a two-layer LSTM-RNN structure can be adopted, and the structure input variable can be a variable-length action series ⁇ X 1 , X 2 , X 3 ,...X n-1 , X n ⁇
  • the variable-length action sequence may specifically be a human body posture image
  • the X n may be any frame of the human body posture image in the human body posture image.
  • the emotion vector corresponding to the facial expression can also be input as another parameter.
  • the emotion vector can be used to represent the emotion type of the actual person, specifically, each human face in the human body posture image
  • An expression can correspond to a different emotion type.
  • the emotion type can be represented by a value between 0 and 1, for example, 1 is used to represent anger, 2 is used to represent surprise, etc., and the human body posture image and the emotion vector
  • it is input into the pre-trained emotional intensity model for processing, and emotional intensity data can be obtained.
  • emotional intensity data can be obtained for example, for the human posture of slightly open arms, if the recognized facial expression is normal, then the emotional strength of calmness can be obtained; but if the recognized facial expression is staring, then the emotional strength of surprise can be obtained.
  • the accuracy of recognition can be improved and correct feedback can be made.
  • a person if a person’s emotions are dissatisfaction, he may frown, and if irritated by improper behavior, he may express strong dissatisfaction by shrugging his shoulders. But if you can detect this emotional fluctuation in time and make adaptive adjustments based on people's reactions, people's dissatisfaction will be alleviated and users will be more accepted.
  • the movement of slightly opening the arms is just a normal posture when a person expresses his thoughts calmly; but if his expression is surprised, it will be much more surprised than the naturally hanging arms. Therefore, in order to enable the robot to obtain a more comprehensive analysis of human body posture actions, the emotion type can be included in the process of emotion intensity estimation.
  • using the pre-trained emotion strength model to process the posture matrix and the obtained emotion vector information to obtain the emotion strength may specifically include: using a sigmoid function to simultaneously perform the input posture matrix and emotion vector information. Process and output the obtained emotional intensity data, which is stored in the blockchain.
  • the posture matrix extracted from the human posture image and the emotion vector information are simultaneously input into the pre-trained emotion intensity model, and the sigmoid function in the model can be called for processing.
  • the sigmoid function can be used to convert any real number Converted to a certain number between 0-1 as the probability. For example, after the sigmoid function processes the posture matrix and emotion vector information, the probability of different emotion intensity data can be obtained, such as anger 93%, less than 5% , Happy 1%, excited 1%, the emotional intensity data with the highest probability can be output.
  • the above-mentioned emotional intensity data may also be stored in a node of a blockchain.
  • the emotional intensity data may also be stored in a node of the blockchain.
  • a blockchain network can be established in advance, and the recording nodes in the blockchain network can be used to record emotional intensity data, the emotional intensity data can be packaged and stored in a new block, and the generated management key Save it in the record node for retrieval and feedback when needed.
  • the emotional intensity data is stored through the blockchain technology, which can greatly ensure the security of the emotional intensity data, and it is easy to retrieve the data, and can improve the efficiency of emotion recognition.
  • the step 102 may specifically include: obtaining Euler angle parameters of each feature point; and determining the posture of each feature point based on the human body static model coordinate system according to the Euler angle parameters matrix.
  • Euler angles indicate that the Euler angles can be used to determine a set of three independent angular parameters of the position of a fixed-point rotating rigid body.
  • the rotational freedom of a joint point can be represented by a set of three angular parameters of rpy.
  • a standardization of the body feature data of different people is realized by inputting a pre-defined human skeleton model.
  • Each feature point in the pre-defined model has a pre-defined coordinate system.
  • the key point extraction technology in OpenPose can be used to determine that each joint is in the human body.
  • the OpenPose human body gesture recognition is an open source library developed by Carnegie Mellon University (CMU) based on convolutional neural networks and supervised learning and using caffe as the framework. It can realize posture estimation of human body movements, facial expressions, and finger movements. It is suitable for single and multiple people, and has excellent robustness.
  • the input is an image
  • the basic model can be VGG19
  • the model output is a matrix that can be represented by the above posture.
  • r, p, y in the matrix can respectively represent the Euler angle parameters of each feature point.
  • the method may further include: performing recognition processing on the acquired facial expression information by using a preset facial recognition algorithm to obtain corresponding emotion vector information.
  • the specific process of the facial recognition algorithm may include: reading facial expression images, using the top of the head as a reference point to estimate the approximate position of facial features, and evenly setting mark points on the contours of each feature part of the face; passing through the center of the brow and the pupils The central axis fitted by the midpoint of the connection line and the three points at the center of the mouth divides the face into two symmetrical parts. Without scaling, translation, or rotation, the image will be adjusted to be symmetrical with respect to the central axis.
  • the facial expression shape model divides the left eye/right eye, left eyebrow/right eyebrow and mouth into different areas, and define these areas as feature candidate areas; For each feature candidate area, the feature vector is extracted by the difference image method. By performing the difference operation between all the image sequences in the image processed in the previous step and the neutral expression image in the database, the average value of the difference value in each feature candidate area is the largest The facial expression feature vector is extracted from the image sequence.
  • the emotion vector data corresponding to the facial expression feature vector is retrieved locally.
  • the emotion vector data may indicate the type of emotion expressed by facial expressions. For example, through the facial expression of frowning, the emotion vector dissatisfaction can be correspondingly obtained.
  • the method may further include: according to the human body posture information, establishing a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system; The coordinates of the nodes are related, and the joint points are determined as the characteristic points of the posture of the human body.
  • the G coordinate system can be a human body static model coordinate system, and the representation method of the coordinate system can be specifically described as follows: "Skeleton tree", the relative relationship between feature points and feature points in the skeleton tree is statically stored as a predefined model; secondly, since everyone's body structure is the same but the skeleton length is different, homogeneous transformation can be introduced
  • the matrix T represents the rigid transformation of different individuals with respect to the corresponding points on the static model. The position of any point can be obtained by matrix multiplication.
  • the method may further include: training an emotional strength model according to the RNN-LSTM model, sample pose data, and preset emotional pose tags.
  • model training pre-training needs to be performed by making a data set or using a data set on the network. For example, for a certain posture, the actual emotion of the observed person needs to be determined and data labeling is completed.
  • the specific process of the model training may include: RNN-based improved long-term and short-term memory modeling. Since many data in real life have both temporal and spatial features, such as human body motion trajectory, continuous frames of video, etc., human body The same is true for posture, each time point corresponds to a set of data, and the data often has certain spatial characteristics. Therefore, in order to carry out classification and prediction work on such a time series, it is necessary to model and extract features in time and space.
  • the commonly used time modeling tool is the Recurrent Neural Network (RNN) correlation model (LSTM). Because of its unique gate structure design and powerful extraction of time series features, it is widely used in forecasting problems and has achieved good results. .
  • the traditional LSTM structure includes three structures: input gate, output gate, forget gate, and a neural node (cell), where the input can be the body's posture representation in the current frame at time t, The output can be a posture descriptor, which is used to describe the type of current posture.
  • a double-layer LSTM-RNN structure is formed by connecting n LSTM structures horizontally, because a continuous image stream is often required when determining the human body posture, so through ⁇ X 1 , X 2 , X 3 ,. . X n-1 ,X n ⁇ to represent such a video stream and serve as the input of the model.
  • the training of the model requires a data set for pre-training, and then a second training is performed by creating a data set to achieve a relatively robust effect.
  • the current posture image stream of a person is recorded through the camera, the posture description matrix is obtained through the key point extraction method of OpenPose, and the current person's emotion type strength is asked to complete the data labeling, and then the training is carried out.
  • the structure of the LSTM unit is still adopted, but the structure of a double-layer LSTM and a fully connected layer are adopted.
  • the double-layer structure can increase the correlation detection of the timing.
  • This application provides an emotional information recognition method. Compared with the prior art inferring human emotions only through facial expressions, this application receives an emotional information recognition request that carries human posture information; using predictions The posture conversion algorithm is assumed to convert the human posture information into a posture matrix containing posture feature points; the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, retrieve and merge Feedback corresponding emotion type. Therefore, the accuracy and efficiency of emotion information recognition can be improved through the dual dimensions of human body posture and facial expression. In addition, this application also uses blockchain technology to store data, which can improve the security of emotional information.
  • an embodiment of the present application provides an emotional information recognition device.
  • the device includes: a receiving unit 21, a conversion unit 22, a processing unit 23 and a feedback unit 24.
  • the receiving unit 21 may be configured to receive an emotion information recognition request, where the emotion information recognition request carries human posture information;
  • the conversion unit 22 may be used to convert the human body posture information into a posture matrix containing posture feature points by using a preset posture conversion algorithm;
  • the processing unit 23 may be configured to process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data;
  • the feedback unit 24 may be used to retrieve and feed back the corresponding emotion type according to the emotion intensity data.
  • processing unit 23 includes:
  • the processing module 231 may be used to process the posture matrix and the obtained emotion vector information using a pre-trained emotion intensity model to obtain the emotion intensity.
  • processing module 231 may be specifically configured to simultaneously process the input posture matrix and emotion vector information by using the sigmoid function, and output the obtained emotion intensity data.
  • processing module 231 may be specifically used to store the emotional intensity data using blockchain technology.
  • the conversion unit 22 includes:
  • the obtaining module 221 may be used to obtain Euler angle parameters of each feature point
  • the determining module 222 may be used to determine the posture matrix of each feature point in the human body static model coordinate system according to the Euler angle parameters.
  • the device further includes:
  • the recognition unit 25 may be used to perform recognition processing on the acquired facial expression information by using a preset facial recognition algorithm to obtain corresponding emotion vector information.
  • the device further includes:
  • the establishing unit 26 may be used to establish, according to the human body posture information, a homogeneous transformation matrix of the human body joint points based on the human body static model coordinate system;
  • the determining unit 27 may be used to determine the coordinates of each joint point by matrix multiplication, and determine the joint point as a feature point of the posture of the human body.
  • the device further includes:
  • the training unit 28 may be used to train the emotional intensity model according to the RNN-LSTM model, sample pose data, and preset emotional pose labels.
  • an embodiment of the present application further provides a storage medium in which at least one executable instruction is stored, and the executable instruction causes the processor to perform the following steps: receiving Emotion information recognition request, said emotion information recognition request carries human body posture information; using a preset posture conversion algorithm to convert the human body posture information into a posture matrix containing posture feature points; according to the preset emotion intensity algorithm The posture matrix is processed to obtain emotion intensity data; according to the emotion intensity data, the corresponding emotion type is retrieved and fed back.
  • the executable instruction can also implement other steps of the method in the foregoing embodiment when executed by the processor, which will not be repeated here.
  • the storage medium involved in the present application may be a computer-readable storage medium, and the storage medium, such as a computer-readable storage medium, may be non-volatile or volatile.
  • an embodiment of the present application also provides a computer device.
  • the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34.
  • the communication interface 34 is used to communicate with other devices, such as network elements such as user terminals or other servers.
  • the processor 31 is configured to execute a program, and specifically can execute relevant steps in the foregoing embodiment of the emotion information recognition method.
  • the program may include program code, and the program code includes computer operation instructions.
  • the processor 31 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
  • ASIC Application Specific Integrated Circuit
  • the one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
  • the memory 33 is used to store programs.
  • the memory 33 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), for example, at least one disk memory.
  • the program can specifically be used to make the processor 31 perform the following operations: receive an emotion information recognition request, the emotion information recognition request carries human posture information; use a preset posture conversion algorithm to convert the human posture information to include posture features Point posture matrix; process the posture matrix according to a preset emotion intensity algorithm to obtain emotion intensity data; retrieve and feed back the corresponding emotion type according to the emotion intensity data.
  • the emotion information recognition request carries human body posture information; use a preset posture conversion algorithm to convert the human body posture information into a posture matrix containing posture feature points;
  • the posture matrix is processed according to a preset emotion intensity algorithm to obtain emotion intensity data; according to the emotion intensity data, the corresponding emotion type is retrieved and fed back. Therefore, the accuracy and efficiency of emotion information recognition can be improved through the dual dimensions of human body posture and facial expression.
  • this application also uses blockchain technology to store data, which can improve the security of emotional information.
  • the blockchain referred to in this application is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.
  • Blockchain essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block.
  • the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
  • modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and/or processes or units are mutually exclusive, any combination can be used to compare all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or methods disclosed in this manner or All the processes or units of the equipment are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
  • the various component embodiments of the present application may be implemented by hardware, or by software modules running on one or more processors, or by a combination of them.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present application.
  • This application can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein.
  • Such a program for implementing the present application may be stored on a computer-readable medium, or may have the form of one or more signals.
  • Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

Sont divulgués un procédé et un appareil de reconnaissance d'informations d'émotion, ainsi qu'un support de stockage et un dispositif informatique, qui se rapportent au domaine technique de l'intelligence artificielle, et visent à traiter respectivement une posture de corps humain et une expression faciale, qui ont été obtenus, dans une matrice de posture et des informations de vecteur d'émotion, et à traiter simultanément, au moyen d'un modèle de reconnaissance d'intensité d'émotion pré-entraîné, la matrice de posture et les informations de vecteur pour obtenir des données d'intensité d'émotion et reconnaître de manière correspondante un type d'émotion, ce qui permet d'améliorer la précision et l'efficacité de la reconnaissance d'informations d'émotion. Le procédé consiste à : recevoir une demande de reconnaissance d'informations d'émotion, la demande de reconnaissance d'informations d'émotion transportant des informations de posture de corps humain; transformer les informations de posture de corps humain en une matrice de posture comprenant des points caractéristiques de posture en utilisant un algorithme de transformation de posture prédéfini; traiter la matrice de posture selon un algorithme d'intensité d'émotion prédéfini de façon à obtenir des données d'intensité d'émotion; et récupérer et renvoyer un type d'émotion correspondant en fonction des données d'intensité d'émotion. La présente demande concerne également la technologie de la chaîne de blocs, les données d'intensité d'émotion pouvant être stockées dans une chaîne de blocs.
PCT/CN2020/111036 2020-04-28 2020-08-25 Procédé et appareil de reconnaissance d'informations d'émotion, support de stockage et dispositif informatique WO2021217973A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010349534.0 2020-04-28
CN202010349534.0A CN111680550A (zh) 2020-04-28 2020-04-28 情感信息识别方法、装置、存储介质及计算机设备

Publications (1)

Publication Number Publication Date
WO2021217973A1 true WO2021217973A1 (fr) 2021-11-04

Family

ID=72452275

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/111036 WO2021217973A1 (fr) 2020-04-28 2020-08-25 Procédé et appareil de reconnaissance d'informations d'émotion, support de stockage et dispositif informatique

Country Status (2)

Country Link
CN (1) CN111680550A (fr)
WO (1) WO2021217973A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863548A (zh) * 2022-03-22 2022-08-05 天津大学 基于人体运动姿态非线性空间特征的情绪识别方法及装置
CN114998834A (zh) * 2022-06-06 2022-09-02 杭州中威电子股份有限公司 一种基于人脸图像以及情感识别的医疗示警系统
CN114863548B (zh) * 2022-03-22 2024-05-31 天津大学 基于人体运动姿态非线性空间特征的情绪识别方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022141894A1 (fr) * 2020-12-31 2022-07-07 苏州源想理念文化发展有限公司 Procédé d'analyse d'émotion caractéristique tridimensionnelle pouvant fusionner l'expression et le mouvement d'un membre
CN113255557B (zh) * 2021-06-08 2023-08-15 苏州优柿心理咨询技术有限公司 一种基于深度学习的视频人群情绪分析方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160095735A (ko) * 2015-02-04 2016-08-12 단국대학교 천안캠퍼스 산학협력단 사용자 얼굴의 복합 다중 감정 인식 방법 및 시스템
CN105868694A (zh) * 2016-03-24 2016-08-17 中国地质大学（武汉） 基于面部表情和眼球动作的双模态情感识别方法及系统
CN108805087A (zh) * 2018-06-14 2018-11-13 南京云思创智信息科技有限公司 基于多模态情绪识别系统的时序语义融合关联判断子系统
CN109145754A (zh) * 2018-07-23 2019-01-04 上海电力学院 融合面部表情及肢体动作三维特征的情绪识别方法
CN110147729A (zh) * 2019-04-16 2019-08-20 深圳壹账通智能科技有限公司 用户情绪识别方法、装置、计算机设备及存储介质
CN111401116A (zh) * 2019-08-13 2020-07-10 南京邮电大学 基于增强卷积和空时lstm网络的双模态情感识别方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106803098A (zh) * 2016-12-28 2017-06-06 南京邮电大学 一种基于语音、表情与姿态的三模态情感识别方法
CN108596039B (zh) * 2018-03-29 2020-05-05 南京邮电大学 一种基于3d卷积神经网络的双模态情感识别方法及系统
CN109684911B (zh) * 2018-10-30 2021-05-11 百度在线网络技术（北京）有限公司 表情识别方法、装置、电子设备及存储介质
CN109815938A (zh) * 2019-02-27 2019-05-28 南京邮电大学 基于多类核典型相关分析的多模态情感特征识别方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20160095735A (ko) * 2015-02-04 2016-08-12 단국대학교 천안캠퍼스 산학협력단 사용자 얼굴의 복합 다중 감정 인식 방법 및 시스템
CN105868694A (zh) * 2016-03-24 2016-08-17 中国地质大学（武汉） 基于面部表情和眼球动作的双模态情感识别方法及系统
CN108805087A (zh) * 2018-06-14 2018-11-13 南京云思创智信息科技有限公司 基于多模态情绪识别系统的时序语义融合关联判断子系统
CN109145754A (zh) * 2018-07-23 2019-01-04 上海电力学院 融合面部表情及肢体动作三维特征的情绪识别方法
CN110147729A (zh) * 2019-04-16 2019-08-20 深圳壹账通智能科技有限公司 用户情绪识别方法、装置、计算机设备及存储介质
CN111401116A (zh) * 2019-08-13 2020-07-10 南京邮电大学 基于增强卷积和空时lstm网络的双模态情感识别方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863548A (zh) * 2022-03-22 2022-08-05 天津大学 基于人体运动姿态非线性空间特征的情绪识别方法及装置
CN114863548B (zh) * 2022-03-22 2024-05-31 天津大学 基于人体运动姿态非线性空间特征的情绪识别方法及装置
CN114998834A (zh) * 2022-06-06 2022-09-02 杭州中威电子股份有限公司 一种基于人脸图像以及情感识别的医疗示警系统

Also Published As

Publication number Publication date
CN111680550A (zh) 2020-09-18

Similar Documents

Publication Publication Date Title
WO2019174439A1 (fr) Procédé et appareil de reconnaissance d'image, terminal et support de mémoire
WO2021217973A1 (fr) Procédé et appareil de reconnaissance d'informations d'émotion, support de stockage et dispositif informatique
CN109948475B (zh) 一种基于骨架特征和深度学习的人体动作识别方法
Storey et al. 3DPalsyNet: A facial palsy grading and motion recognition framework using fully 3D convolutional neural networks
CN108363973B (zh) 一种无约束的3d表情迁移方法
Deng et al. MVF-Net: A multi-view fusion network for event-based object classification
AU2014253687B2 (en) System and method of tracking an object
CN110210426B (zh) 基于注意力机制从单幅彩色图像进行手部姿态估计的方法
CN112329525A (zh) 一种基于时空图卷积神经网络的手势识别方法和装置
Rajalakshmi et al. Multi-semantic discriminative feature learning for sign gesture recognition using hybrid deep neural architecture
Sabater et al. Domain and view-point agnostic hand action recognition
Neverova Deep learning for human motion analysis
CN112906520A (zh) 一种基于姿态编码的动作识别方法及装置
Podder et al. Time efficient real time facial expression recognition with CNN and transfer learning
Wan et al. Midoriko chatbot: LSTM-based emotional 3D avatar
Pattar et al. Intention and engagement recognition for personalized human-robot interaction, an integrated and deep learning approach
Wu et al. Limb pose aware networks for monocular 3D pose estimation
Rastgoo et al. Word separation in continuous sign language using isolated signs and post-processing
Miah et al. Sign Language Recognition using Graph and General Deep Neural Network Based on Large Scale Dataset
Bayegizova et al. EFFECTIVENESS OF THE USE OF ALGORITHMS AND METHODS OF ARTIFICIAL TECHNOLOGIES FOR SIGN LANGUAGE RECOGNITION FOR PEOPLE WITH DISABILITIES.
JP2022531828A (ja) 手動作の対称性の認識方法及びシステム
Aiman et al. Angle based hand gesture recognition using graph convolutional network
Senanayaka et al. Continuous American Sign Language Recognition Using Computer Vision And Deep Learning Technologies
CN111178141B (zh) 一种基于注意力机制的lstm人体行为识别方法
JP2022075329A (ja) 認識システム、認識方法、プログラム、学習方法、学習済みモデル、蒸留モデル、及び、学習用データセット生成方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20933591

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20933591

Country of ref document: EP

Kind code of ref document: A1