CN112488219A - Mood consolation method and system based on GRU and mobile terminal - Google Patents
Mood consolation method and system based on GRU and mobile terminal Download PDFInfo
- Publication number
- CN112488219A CN112488219A CN202011417391.9A CN202011417391A CN112488219A CN 112488219 A CN112488219 A CN 112488219A CN 202011417391 A CN202011417391 A CN 202011417391A CN 112488219 A CN112488219 A CN 112488219A
- Authority
- CN
- China
- Prior art keywords
- emotion
- gru
- mobile terminal
- user
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000036651 mood Effects 0.000 title description 2
- 230000008451 emotion Effects 0.000 claims abstract description 74
- 230000002996 emotional effect Effects 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 18
- 230000008921 facial expression Effects 0.000 claims abstract description 10
- 230000000694 effects Effects 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 6
- 230000008909 emotion recognition Effects 0.000 claims description 22
- 239000013598 vector Substances 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 4
- 230000006399 behavior Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 230000001815 facial effect Effects 0.000 claims description 3
- 238000003672 processing method Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000004880 explosion Methods 0.000 abstract description 2
- 230000008034 disappearance Effects 0.000 abstract 1
- 238000013461 design Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Processing Or Creating Images (AREA)
Abstract
A method, a system and a mobile terminal for consolation of emotion based on GRU belong to the technical field of intelligent equipment, and comprise the following steps: recording the facial expressions and the sounds into a system, and training the received data so as to realize the recognition of the emotion of the user; according to the recognized emotion, the terminal makes a preset comfort action; the mobile terminal records the time, the adjusting mode, the adjusting duration and other information of each emotion adjustment of the user; the obtained information is brought into a GRU network for relearning; data reports are generated periodically. The invention has the beneficial effects that: the double-layer direction GRU is adopted to respectively simulate human auditory and visual processing paths to process the emotional information of voice and facial expressions, the GRU can overcome the problems of gradient disappearance and explosion during RNN modeling, the training time is shorter than that of LSTM, and the overfitting problem is less. After the attention mechanism is introduced, the influence weight of important time sequence characteristics can be improved, non-important time sequence characteristics are restrained, and the classification effect of the model is improved.
Description
Technical Field
The invention belongs to the technical field of intelligent equipment, and particularly relates to a method and a system for emotion comfort based on GRU (generalized gre unit) voice and image recognition.
Background
With the development of society, the rhythm of life of people is faster and faster. Sadness emotion at ordinary times is more and more reluctant, and the application field of expression recognition is wide at present, but the emotion recognition is rarely applied to human emotion directly. Tragedies that occur because of psychological factors are coming out endlessly, especially in student groups.
People generally carry the mobile terminal when going out, the time for using the mobile terminal is also in the trend of increasing year by year, and the interaction between the mobile terminal and the mobile terminal also becomes the main mode for expressing the view of people and releasing the emotion.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method, a system and a mobile terminal for consolation based on the emotion of a GRU (generalized gre). A camera of the mobile terminal is used for taking a picture in real time, a microphone of the mobile terminal is used for recording the picture, the picture is put into a long-time memory GRU network for training, so that the emotion of a user is classified, and the mobile terminal is controlled to consolation the user by using various optional methods according to a judgment result.
A GRU-based emotional comfort method comprises the following steps:
step 1, inputting facial expressions and voices of a person into a system, and putting collected picture and voice data into a GRU network algorithm for training so as to realize the recognition of the emotion of a user;
step 2, the terminal performs a preset comfort action according to the recognized emotion of the user;
step 3, the mobile terminal records the time of each emotion adjustment of the user, the mode used for adjusting the emotion, the adjustment duration and the evaluation information of the user on the adjustment effect;
step 4, bringing the information fed back in the step 3 into a GRU network for relearning so as to adapt to behavior preferences of different users;
and 5, periodically generating a data report.
Preferably, the emotion recognition in step 1 specifically includes the following steps:
step 11, inputting video and audio;
step 12, preprocessing the audio and extracting 43-dimensional effective features; processing the video to extract 26-dimensional effective features of the video;
step 13, carrying the effective characteristics of the audio and the video into a GRU network for training;
and step 14, carrying out decision layer fusion algorithm to identify the emotion of the user.
Preferably, the video processing method in step 12 includes the following steps:
step 121: extracting image frames, namely extracting one picture every 3 frames;
step 122: extracting 68 coordinates of the facial feature points from the step 121 by using a Dlib library;
step 123: on the basis of the coordinates of the 68 feature points, the distance length between 26 two points is selected as an expression feature;
step 124: the 26-dimensional features are fed into the GRU network training and testing.
Preferably, the extraction of the voice effective features in step 12 includes the following steps:
step 125: for the preprocessing work of the audio, the window length is set to be 0.025s respectively, and the time interval for extracting the speech emotion characteristics is 0.01 s;
step 126: and performing feature extraction, wherein 43-dimensional feature vectors for representing the speech emotion are extracted in total, and the 13-dimensional MFCC features, the 2-dimensional MFCC dynamic difference parameters comprise MFCC1 order difference and 2 order difference respectively, and the 26-dimensional Fbank features and the 2-dimensional standard differences comprise MFCC and Fbank standard differences respectively.
Preferably, the decision layer fusion algorithm in step 14 includes the following steps:
step 141: and splicing the 43-dimensional feature vector extracted by the voice and the 26-dimensional feature vector extracted by the video into a 59-dimensional emotion feature vector and standardizing the 59-dimensional emotion feature vector.
Step 142: the standardized features are sent to a GRU network for training and testing;
step 143: and integrating the voice and facial expression emotion recognition results output by the GRU in a weighting mode.
Preferably, the emotion is classified into 6 types through emotion recognition in step 2, and the mobile terminal will respectively react differently according to different emotion types:
if the emotion is identified as happy, the mobile terminal does not make any reaction;
if the emotion recognition is surprise, the mobile terminal automatically pops up a webpage search bar to enable the user to search for things which are surprised by the user;
if the emotion is recognized as fear, disgust, sadness or angry emotion, the mobile terminal plays light music or plays a funny video, if a certain time passes, the emotion is recognized to be still fear through emotion recognition, and the mobile terminal automatically contacts preset contacts to seek artificial psychological comfort;
preferably, the mobile terminal records the mode of emotion adjustment and the manual evaluation information of the user in each time in step 4, the user learns through the GRU network, the relationship among the mode of emotion adjustment, the type of emotion, the adjustment effect and the adjustment duration of each user is determined in a personalized manner, and the related data is stored in the mobile terminal for the user to call and learn again when the user uses the mobile terminal next time.
The GRU-based emotion comfort system realizes the steps of the comfort method.
A GRU based emotional comfort mobile terminal comprising a memory, a processor, a camera, a screen, a speaker, a microphone, a communication device and a GRU based emotional comfort program stored on the memory and executable on the processor, the GRU based emotional comfort program when executed by the processor implementing the steps of the GRU based emotional comfort method as described above.
The invention has the beneficial effects that:
the audio/video emotion recognition in emotion calculation has important application value for deep level cognition in the fields of human-computer interaction and the like, in order to overcome the problem that the recognition accuracy of a single-modal model depends on emotion types, a multi-modal emotion recognition model based on a GRU network is provided, the emotion information of voice and facial expressions is processed by respectively simulating human auditory and visual processing paths by adopting a double-layer direction GRU, and the GRU can overcome the problems of gradient loss and explosion in RNN modeling, is shorter than LSTM training time and has fewer overfitting problems. After the attention mechanism is introduced, the influence weight of important time sequence characteristics can be improved, non-important time sequence characteristics are restrained, and the classification effect of the model is improved. Meanwhile, the problem that the traditional discrete emotion six-classification method cannot measure the degree and has the problems of similar appearance and simultaneous coexistence of multiple emotions is considered.
Drawings
FIG. 1 is an overall flow chart of a GRU-based emotional comfort method of the present invention;
fig. 2 is a schematic flow chart of emotion recognition.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a GRU-based emotion comfort method, which comprises the following steps:
step 1, inputting facial expressions and voices of a person into a system, and putting collected picture and voice data into a GRU network algorithm for training so as to realize the recognition of the emotion of a user;
step 2, the terminal performs a preset comfort action according to the recognized emotion of the user;
step 3, the mobile terminal records the time of each emotion adjustment of the user, the mode used for adjusting the emotion, the adjustment duration and the evaluation information of the user on the adjustment effect;
step 4, bringing the information fed back in the step 3 into a GRU network for relearning so as to adapt to behavior preferences of different users;
and 5, periodically generating a data report.
Preferably, the emotion recognition in step 1 specifically includes the following steps:
step 11, inputting video and audio;
step 12, preprocessing the audio and extracting 43-dimensional effective features; processing the video to extract 26-dimensional effective features of the video;
step 13, carrying the effective characteristics of the audio and the video into a GRU network for training;
and step 14, carrying out decision layer fusion algorithm to identify the emotion of the user.
Preferably, the video processing method in step 12 includes the following steps:
step 121: extracting image frames, namely extracting one picture every 3 frames;
step 122: extracting 68 coordinates of the facial feature points from the step 121 by using a Dlib library;
step 123: on the basis of the coordinates of the 68 feature points, the distance length between 26 two points is selected as an expression feature;
step 124: the 26-dimensional features are fed into the GRU network training and testing.
Preferably, the extraction of the voice effective features in step 12 includes the following steps:
step 125: for the preprocessing work of the audio, the window length is set to be 0.025s respectively, and the time interval for extracting the speech emotion characteristics is 0.01 s;
step 126: and performing feature extraction, wherein 43-dimensional feature vectors for representing the speech emotion are extracted in total, and the 13-dimensional MFCC features, the 2-dimensional MFCC dynamic difference parameters comprise MFCC1 order difference and 2 order difference respectively, and the 26-dimensional Fbank features and the 2-dimensional standard differences comprise MFCC and Fbank standard differences respectively.
Preferably, the decision layer fusion algorithm in step 14 includes the following steps:
step 141: and splicing the 43-dimensional feature vector extracted by the voice and the 26-dimensional feature vector extracted by the video into a 59-dimensional emotion feature vector and standardizing the 59-dimensional emotion feature vector.
Step 142: the standardized features are sent to a GRU network for training and testing;
step 143: and integrating the voice and facial expression emotion recognition results output by the GRU in a weighting mode.
Preferably, the emotion is classified into 6 types through emotion recognition in step 2, and the mobile terminal will respectively react differently according to different emotion types:
if the emotion is identified as happy, the mobile terminal does not make any reaction;
if the emotion recognition is surprise, the mobile terminal automatically pops up a webpage search bar to enable the user to search for things which are surprised by the user;
if the emotion is recognized as fear, disgust, sadness or angry emotion, the mobile terminal plays light music or plays a funny video, if a certain time passes, the emotion is recognized to be still fear through emotion recognition, and the mobile terminal automatically contacts preset contacts to seek artificial psychological comfort;
preferably, the mobile terminal records the mode of emotion adjustment and the manual evaluation information of the user in each time in step 4, the user learns through the GRU network, the relationship among the mode of emotion adjustment, the type of emotion, the adjustment effect and the adjustment duration of each user is determined in a personalized manner, and the related data is stored in the mobile terminal for the user to call and learn again when the user uses the mobile terminal next time.
The multi-modal emotion recognition method provided by the invention mainly trains audio data and video data respectively by relying on a GRU network in a double-layer direction, and the double-layer direction respectively simulates human auditory and visual processing paths to process voice and facial expression video information. Fig. 2 is an emotion recognition module of the present design. In the overall design, in order to improve the training and test training efficiency, the design extracts few effective features, an audio channel extracts 43-dimensional effective features, a video channel extracts 26-dimensional effective features, and the total effective features are only 69-dimensional, so that the model can complete GRU multi-mode emotion recognition which is performed in real time and has excellent performance. The model selects a decision layer fusion method on feature fusion, so that the emotion is classified finally.
The GRU-based emotion comfort system realizes the steps of the comfort method.
A GRU based emotional comfort mobile terminal comprising a memory, a processor, a camera, a screen, a speaker, a microphone, a communication device and a GRU based emotional comfort program stored on the memory and executable on the processor, the GRU based emotional comfort program when executed by the processor implementing the steps of the GRU based emotional comfort method as described above.
Claims (9)
1. A GRU-based emotional comfort method is characterized by comprising the following steps:
step 1, inputting facial expressions and voices of a person into a system, and putting collected picture and voice data into a GRU network algorithm for training so as to realize the recognition of the emotion of a user;
step 2, the terminal performs a preset comfort action according to the recognized emotion of the user;
step 3, the mobile terminal records the time of each emotion adjustment of the user, the mode used for adjusting the emotion, the adjustment duration and the evaluation information of the user on the adjustment effect;
step 4, bringing the information fed back in the step 3 into a GRU network for relearning so as to adapt to behavior preferences of different users;
and 5, periodically generating a data report.
2. The GRU-based emotional comfort method according to claim 1, wherein the emotion recognition in step 1 specifically comprises the following steps:
step 11, inputting video and audio;
step 12, preprocessing the audio and extracting 43-dimensional effective features; processing the video to extract 26-dimensional effective features of the video;
step 13, carrying the effective characteristics of the audio and the video into a GRU network for training;
and step 14, carrying out decision layer fusion algorithm to identify the emotion of the user.
3. A GRU-based emotional comfort method according to claim 2, wherein the processing method of the video in step 12 comprises the following steps:
step 121: extracting image frames, namely extracting one picture every 3 frames;
step 122: extracting 68 coordinates of the facial feature points from the step 121 by using a Dlib library;
step 123: on the basis of the coordinates of the 68 feature points, the distance length between 26 two points is selected as an expression feature;
step 124: the 26-dimensional features are fed into the GRU network training and testing.
4. The GRU-based emotional comfort method of claim 2, wherein the extraction of the speech-active features in step 12 comprises the steps of:
step 125: for the preprocessing work of the audio, the window length is set to be 0.025s respectively, and the time interval for extracting the speech emotion characteristics is 0.01 s;
step 126: and performing feature extraction, wherein 43-dimensional feature vectors for representing the speech emotion are extracted in total, and the 13-dimensional MFCC features, the 2-dimensional MFCC dynamic difference parameters comprise MFCC1 order difference and 2 order difference respectively, and the 26-dimensional Fbank features and the 2-dimensional standard differences comprise MFCC and Fbank standard differences respectively.
5. The GRU-based emotional comfort method of claim 2, wherein the decision layer fusion algorithm in step 14 comprises the following steps:
step 141: splicing the 43-dimensional feature vector extracted by the voice and the 26-dimensional feature vector extracted by the video into 59-dimensional emotion feature vectors and standardizing the 59-dimensional emotion feature vectors;
step 142: the standardized features are sent to a GRU network for training and testing;
step 143: and integrating the voice and facial expression emotion recognition results output by the GRU in a weighting mode.
6. The GRU-based emotion comforting method according to claim 1, wherein the emotions are classified into 6 types through emotion recognition in step 2, and the mobile terminal will respectively react differently according to different emotion types:
if the emotion is identified as happy, the mobile terminal does not make any reaction;
if the emotion recognition is surprise, the mobile terminal automatically pops up a webpage search bar to enable the user to search for things which are surprised by the user;
if the emotion is recognized as fear, disgust, sadness or anger, the mobile terminal plays a happy music or a funny video, if a certain time passes, the emotion is recognized to be still fear through emotion recognition, and the mobile terminal automatically contacts a preset contact person to seek artificial psychological comfort.
7. The method for consolation of emotion based on GRU of claim 1, wherein in step 4 the mobile terminal records the mode of emotion adjustment each time and the manual evaluation information of the user, learns through the GRU network, determines the relationship between the mode of emotion adjustment, the type of emotion, the effect of adjustment and the duration of adjustment for each user in a personalized manner, and stores the relevant data in the mobile terminal for the user to call and relearn when using next time.
8. A GRU based emotional comfort system, characterized in that it implements the steps of the GRU based emotional comfort method of claims 1 to 7.
9. A GRU based emotional comfort mobile terminal, characterized in that the mobile terminal comprises a memory, a processor, a camera, a screen, a speaker, a microphone, communication means and a GRU based emotional comfort program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the GRU based emotional comfort method of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011417391.9A CN112488219A (en) | 2020-12-07 | 2020-12-07 | Mood consolation method and system based on GRU and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011417391.9A CN112488219A (en) | 2020-12-07 | 2020-12-07 | Mood consolation method and system based on GRU and mobile terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112488219A true CN112488219A (en) | 2021-03-12 |
Family
ID=74939932
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011417391.9A Pending CN112488219A (en) | 2020-12-07 | 2020-12-07 | Mood consolation method and system based on GRU and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112488219A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130337420A1 (en) * | 2012-06-19 | 2013-12-19 | International Business Machines Corporation | Recognition and Feedback of Facial and Vocal Emotions |
| CN109409296A (en) * | 2018-10-30 | 2019-03-01 | 河北工业大学 | The video feeling recognition methods that facial expression recognition and speech emotion recognition are merged |
| CN109451356A (en) * | 2018-12-20 | 2019-03-08 | 珠海市微半导体有限公司 | A kind of intelligent mobile robot, automatic order method, device and chip |
| CN110110653A (en) * | 2019-04-30 | 2019-08-09 | 上海迥灵信息技术有限公司 | The Emotion identification method, apparatus and storage medium of multiple features fusion |
| CN111275085A (en) * | 2020-01-15 | 2020-06-12 | 重庆邮电大学 | Online short video multi-modal emotion recognition method based on attention fusion |
| CN111368649A (en) * | 2020-02-17 | 2020-07-03 | 杭州电子科技大学 | Emotion perception method operating in raspberry pie |
-
2020
- 2020-12-07 CN CN202011417391.9A patent/CN112488219A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130337420A1 (en) * | 2012-06-19 | 2013-12-19 | International Business Machines Corporation | Recognition and Feedback of Facial and Vocal Emotions |
| CN109409296A (en) * | 2018-10-30 | 2019-03-01 | 河北工业大学 | The video feeling recognition methods that facial expression recognition and speech emotion recognition are merged |
| CN109451356A (en) * | 2018-12-20 | 2019-03-08 | 珠海市微半导体有限公司 | A kind of intelligent mobile robot, automatic order method, device and chip |
| CN110110653A (en) * | 2019-04-30 | 2019-08-09 | 上海迥灵信息技术有限公司 | The Emotion identification method, apparatus and storage medium of multiple features fusion |
| CN111275085A (en) * | 2020-01-15 | 2020-06-12 | 重庆邮电大学 | Online short video multi-modal emotion recognition method based on attention fusion |
| CN111368649A (en) * | 2020-02-17 | 2020-07-03 | 杭州电子科技大学 | Emotion perception method operating in raspberry pie |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564942B (en) | Voice emotion recognition method and system based on adjustable sensitivity | |
| CN108717856B (en) | Speech emotion recognition method based on multi-scale deep convolution cyclic neural network | |
| CN111415677B (en) | Method, apparatus, device and medium for generating video | |
| CN110570873B (en) | Voiceprint wake-up method and device, computer equipment and storage medium | |
| CN104538043A (en) | Real-time emotion reminder for call | |
| CN108520741A (en) | A kind of whispering voice restoration methods, device, equipment and readable storage medium storing program for executing | |
| CN110516696A (en) | It is a kind of that emotion identification method is merged based on the adaptive weighting bimodal of voice and expression | |
| CN110610534B (en) | Automatic mouth shape animation generation method based on Actor-Critic algorithm | |
| CN107993665A (en) | Spokesman role determines method, intelligent meeting method and system in multi-conference scene | |
| WO2020253128A1 (en) | Voice recognition-based communication service method, apparatus, computer device, and storage medium | |
| BRPI0904540B1 (en) | method for animating faces / heads / virtual characters via voice processing | |
| CN110310647A (en) | A kind of speech identity feature extractor, classifier training method and relevant device | |
| WO2019160100A1 (en) | Nonverbal information generation device, nonverbal information generation model learning device, method, and program | |
| KR101738142B1 (en) | System for generating digital life based on emotion and controlling method therefore | |
| CN109558935A (en) | Emotion recognition and exchange method and system based on deep learning | |
| CN109542389B (en) | Sound effect control method and system for multi-mode story content output | |
| WO2019160105A1 (en) | Nonverbal information generation device, nonverbal information generation model learning device, method, and program | |
| WO2019160090A1 (en) | Nonverbal information generation device, method, and program | |
| CN115187704A (en) | Virtual anchor generation method, device, equipment and storage medium | |
| CN113238654A (en) | Multi-modal based reactive response generation | |
| CN114254096A (en) | Multi-mode emotion prediction method and system based on interactive robot conversation | |
| Gomes et al. | i-vector algorithm with Gaussian Mixture Model for efficient speech emotion recognition | |
| KR20190125668A (en) | Apparatus and method for analyzing emotional status of pet | |
| CN109961152B (en) | Personalized interaction method and system of virtual idol, terminal equipment and storage medium | |
| WO2019160104A1 (en) | Nonverbal information generation device, nonverbal information generation model learning device, method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |