CN113208635A - Emotion electroencephalogram signal induction method based on conversation - Google Patents
Emotion electroencephalogram signal induction method based on conversation Download PDFInfo
- Publication number
- CN113208635A CN113208635A CN202110471023.0A CN202110471023A CN113208635A CN 113208635 A CN113208635 A CN 113208635A CN 202110471023 A CN202110471023 A CN 202110471023A CN 113208635 A CN113208635 A CN 113208635A
- Authority
- CN
- China
- Prior art keywords
- emotion
- voice
- conversation
- participant
- experiment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7271—Specific aspects of physiological measurement analysis
Abstract
The invention relates to an emotion electroencephalogram signal induction method based on conversation. The method mainly solves the technical problem that an isolated emotion voice induced electroencephalogram signal is not suitable for actual emotion requirements in the existing method for inducing the emotion electroencephalogram signal. The technical scheme of the invention is as follows: the method comprises the following specific steps: 1) collection of affective material; 2) experiment design, namely, a chat mode of 'WeChat' voice is adopted, so that participants simulate a chat process in a real scene through a role playing mode and a role played by a computer, the real emotion of the participants is induced, and emotion electroencephalogram signals consistent with emotion inducing material labels are obtained; 3) the experiment inducing and collecting adopts a conversation mode combining voice and characters, before the experiment begins, a participant sits still in front of a computer and is provided with an electroencephalogram collecting cap, and when the participant hears voice emitted by a role played by the computer once, the participant carries out emotion dimension scoring on the heard emotion voice signals.
Description
Technical Field
The invention belongs to the technical field of brain science and emotion calculation, and particularly relates to an emotion electroencephalogram signal induction method based on conversation.
Background
The emotional electroencephalogram signals can be induced by various modes, so that the emotional electroencephalogram signals have great difference in the aspects of emotional types, marking schemes and the like. The choice of the inducing mode is mainly based on visual, auditory and audio-visual perception induction at present. For voice induction, the presentation mode is different from that of a visual signal, and the voice induction needs to be presented sequentially according to the time sequence, so that only after audio information is completely broadcast, the human brain can correctly understand the content expressed by the audio signal. In the emotion voice inducing material, acoustic parameters (fundamental frequency, intensity, duration and the like) of different voices are generally difficult to unify, and different documents selecting different emotion inducing materials can make the contrast of experimental results difficult to determine, so that the cognitive analysis result of specific emotions in the voices is not uniform, in addition, in the selection of the emotion materials, the number and the types of the selected emotion types are different, and a plurality of researchers adopt discrete emotion models, such as: type 2 emotions (happy, sad; positive, negative), type 3 emotions (happy, angry, sad), type 4 emotions (angry, aversion, fear, happy), type 6 emotions (angry, aversion, fear, happy, sad, surprise), and the like. Although simple and easy to understand, the discrete emotion model has great limitation in describing emotion capacity, and the number of categories of emotion classification of the discrete emotion model is difficult to determine because human emotion is very complex, and the emotion cannot be contained in all emotion states no matter how many categories the emotion is classified into, and in addition, different translations such as 'Happy' can exist between the same emotion words and can be translated into 'Glad' and 'Happy' due to differences of regions and cultures where people are located. The continuous emotion dimensionality theory does not have the problem, the relation among different emotion categories can be described in multiple directions from multiple angles, and if a wide DEAP database is used at present, emotion electroencephalogram signals are labeled from four continuous emotion dimensionalities (VAD dimensionality and likeness). However, the continuous dimension has the problem of difficult comprehension in manual labeling, such as a three-dimensional continuous emotion VAD (V-Valence (Valence); A-wakefulness (Arousal); D-control (Dominance)) model, and the experimental participants have difficulty in literally understanding the emotional meaning that the model represents. Finally, for voice induction, the electroencephalogram signals based on emotion voice are induced by adopting isolated single sentences or single words, even if the same emotion voice is selected to induce the participants, the information obtained by different experimental participants is not completely the same due to lack of context and transmission and transition of emotion information, because different emotion voices are isolated and presented to the participants, the participants are passive when receiving different emotion voices, and lack of participation in the experimental process, which is different from the cognitive state of the emotion voice in a real scene, the needed electroencephalogram signals are difficult to obtain. In fact, emotional voice exists in a conversation between two or more persons in a certain scene, and in real life, the induction of electroencephalogram signals by using isolated emotional voice is not suitable for actual emotional requirements.
Therefore, aiming at the problem, in order to obtain more real and effective electroencephalogram signals, the method adopts a Chinese language 'conversation' mode to induce the participants, the experimental process is similar to a 'WeChat' voice conversation mode, conversation scene information is added as experimental background introduction, the participants play a specific role in conversation, actively participate in the emotion conversation process to induce the emotion of the participants, meanwhile, emotion inducing materials are labeled in the data acquisition process, emotion electroencephalogram data consistent with the labeled emotion are acquired, and the data can be used for analyzing and identifying the emotion electroencephalogram signals and exploring the brain cognitive process of emotion voice in the real conversation scene.
Disclosure of Invention
The invention aims to solve the technical problem that an isolated emotion voice induced electroencephalogram signal is not suitable for actual emotion requirements in the existing method for inducing the emotion electroencephalogram signal, and provides a method for inducing the emotion electroencephalogram signal based on conversation.
In order to solve the technical problems, the invention adopts the technical scheme that:
a dialogue-based emotion electroencephalogram signal induction method comprises the following specific steps:
1) collection of Emotion inducing materials
The collection of the emotion inducing materials is to collect voice short dialogs with strong emotion colors from network videos or audio novels, collect a plurality of voice short dialogs according to the number of scenes, the number of the dialogs and the voice duration factor, screen the voice short dialogs according to the content and the quality of the voice, encode the screened voice one by one, complete the emotion marking of the voice in the acquisition process of electroencephalogram signals, and obtain emotion voice data with continuous dimensionality and discrete dimensionality;
2) design of experiments
The experimental design is that a chat mode of WeChat voice is adopted, the experimental design is respectively designed into a boy group and a girl group according to the gender of participants, the participants simulate the chat process in a real scene through the role playing mode and the role played by a computer, the real emotion of the participants is induced, and emotion electroencephalogram signals consistent with the emotion induction material labels are obtained;
3) experimental Induction and Collection
The experiment induction and collection adopts a dialogue mode combining voice and characters for the experiment, the role substituted by the participant is set as character content input, and the role played by a computer interacting with the participant is set as voice input; before the experiment begins, a participant sits in front of a computer and is provided with a brain electricity collecting cap, in the experiment process, when the participant hears voice sent by the role played by the computer once, the computer screen can automatically present an emotion dimension scoring table once, the participant needs to score emotion dimensions of the heard emotion voice signals, and the rest can be done until the experiment is finished.
Further, the selection is carried out according to the content and the quality of the voice, the voice needs to be selected from evaluation factors and grading levels, and emotion dialogue groups with emotion expression intensity, situational awareness, definition, naturalness, fluency and background cleanliness all more than three are selected as emotion inducing materials, wherein the grading levels are good, medium, poor and poor, so that the quality of an emotion voice inducing database is ensured and preparation is made for collecting effective emotion electroencephalogram data; the good, medium, poor and bad scores are 5, 4, 3, 2 and 1 respectively.
Furthermore, the emotion marking is based on a human body model self-evaluation method (SAM), a VAD emotion dimensionality scoring table with text explanation is adopted to continuously and discretely mark the emotion in the voice, the text explanation is used as discrete emotion marking, and the continuous emotion marking and the discrete emotion marking are carried out in the electroencephalogram signal acquisition process.
Further, the value of the VAD emotion dimension scoring table is 1-9 points.
Compared with the prior art, the invention has the beneficial effects that:
1. the situation that the emotion is not sufficient induced by a passive induction mode of isolating the emotion when electroencephalogram signals are collected is solved in the past through a mode of 'WeChat' voice conversation;
2. the emotional state of the participant is quickly induced in a role playing mode, so that the emotional expression of the participant is more real and natural;
3. by labeling the emotion-inducing material in the acquisition process, a multi-mode emotion-inducing electroencephalogram signal consistent with the emotion-inducing material label can be obtained, and better data support is provided for emotion calculation and research in related fields.
Drawings
FIG. 1 is a flow chart of the experimental design of the present invention;
in the figure: (a) the participants were females and (b) the participants were males.
Detailed Description
The present invention will be further described with reference to the following examples.
In this embodiment, an emotion electroencephalogram signal induction method based on a dialog includes the specific steps of:
1) collection of Emotion inducing materials
In real life, emotional voice often occurs in a conversation between two or more people in a certain scene, such as quarrel, debate, video conference, WeChat voice chat, and the like. In order to enrich the content of emotional voice, selecting voice short dialogs with stronger emotional colors collected from network videos or audio novels to obtain emotional voice inducing data based on a Chinese dialogue mode, selecting short dialogs under not less than 20 different scenes, selecting the number of the short dialogs in each scene to be within 50 sentences, wherein the voice duration of each sentence is less than or equal to 3s, numbering the voice short dialogs and each sentence of emotional voice in each scene, and continuously and discretely marking the emotion in the voice by adopting a VAD emotion dimensionality scoring table with character explanation in the acquisition process of electroencephalogram signals to obtain emotion voice data with continuous dimensionality and discrete dimensionality; in addition, before the experiment begins, emotion voice conversation contents are primarily screened, the emotion voice conversation contents are mainly screened from evaluation factors and grading levels, emotion conversation groups with emotion expression intensity, situational awareness, definition, naturalness, fluency and background cleanliness more than three minutes are selected as emotion inducing materials, and grading levels are good (5 grades), good (4 grades), medium (3 grades), poor (2 grades) and poor (1 grades), so that the quality of an emotion voice database is guaranteed, and effective emotion electroencephalogram data are prepared for collection.
2) Design of experiments
Performing emotion induction on an experiment participant by adopting an emotion voice conversation mode, intercepting the collected voice conversation again by taking sentences as units by adopting a WeChat voice conversation mode, and embedding the intercepted single sentences into stimulus presentation software according to an original conversation sequence, wherein the emotion voice conversation mode is shown in figure 1; in order to collect emotional electroencephalogram signals close to real environment, a 'chat' background introduction is added for each short conversation before the experiment starts, meanwhile, two different experiments of a boy group and a girl group are designed respectively according to the gender of participants, for example, when the participants are male, the participants can play husband, a computer plays wife, the experiment task is to simulate a husband and wife noise process, so that the participants induce real emotion of the participants through the role playing task, and finally, the emotional electroencephalogram signals consistent with the emotion inducing material labels are obtained.
3) Experimental Induction and Collection
The experiment adopts a dialogue mode combining voice and text, and sets the role substituted by the participants as text content input and the role played by a computer interacting with the participants as voice input. Before the experiment begins, a participant needs to sit in front of a computer with a brain electricity collecting cap, and in the experiment process, when the participant hears role voice played by the computer once, a computer screen can automatically present a VAD emotion dimension scoring table, the participant needs to score emotion dimensions of the heard voice signals, and the rest can be done until the experiment is finished.
TABLE 1 VAD Emotion dimension score Table
The emotional voice is scored on VAD continuous dimension by adopting a human body model self-evaluation method (SAM), and the score is 1-9. In order to enable participants to better understand the meaning of VAD emotion dimension, the invention adds a text explanation on the basis of SAM scale, and the text explanation can be used as a discrete emotion label, as shown in Table 1. Experimenters respectively mark and score feelings in three dimensions of titer degree V, arousal degree A and dominance degree D. Such as: the titer V: let you feel pleasantness or unpleasantness, the higher the pleasantness, the closer to 9 the score, the lower the pleasantness, the closer to 1 the score, where score values 2, 4, 6, 8 indicate that the participant can make finer determinations and selections between the two emotions, through which scale emotion-inducing material and emotion electroencephalograms with continuous and discrete labels can be collected.
Claims (4)
1. A dialogue-based emotion electroencephalogram signal induction method is characterized by comprising the following specific steps:
1) collection of Emotion inducing materials
The collection of the emotion inducing materials is to collect voice short dialogs with strong emotion colors from network videos or audio novels, collect a plurality of voice short dialogs according to the number of scenes, the number of the dialogs and the voice duration factor, screen the voice short dialogs according to the content and the quality of the voice, encode the screened voice one by one, complete the emotion marking of the voice in the acquisition process of electroencephalogram signals, and obtain emotion voice data with continuous dimensionality and discrete dimensionality;
2) design of experiments
The experimental design is that a chat mode of WeChat voice is adopted, the experimental design is respectively designed into a boy group and a girl group according to the gender of participants, the participants simulate the chat process in a real scene through the role playing mode and the role played by a computer, the real emotion of the participants is induced, and emotion electroencephalogram signals consistent with the emotion induction material labels are obtained;
3) experimental Induction and Collection
The experiment induction and collection adopts a dialogue mode combining voice and characters for the experiment, the role substituted by the participant is set as character content input, and the role played by a computer interacting with the participant is set as voice input; before the experiment begins, a participant sits in front of a computer and is provided with a brain electricity collecting cap, in the experiment process, when the participant hears voice sent by the role played by the computer once, the computer screen can automatically present an emotion dimension scoring table once, the participant needs to score emotion dimensions of the heard emotion voice signals, and the rest can be done until the experiment is finished.
2. The emotion electroencephalogram signal induction method based on conversation, as claimed in claim 1, wherein: the method comprises the steps of screening according to the content and quality of voice, screening from evaluation factors and grading levels, selecting emotion dialogue groups with emotion expression intensity, situational awareness, definition, naturalness, fluency and background cleanliness more than three minutes as emotion inducing materials, wherein the grading levels are good, medium, poor and poor so as to ensure the quality of an emotion voice inducing database and prepare for collecting effective emotion electroencephalogram data; the good, medium, poor and bad scores are 5, 4, 3, 2 and 1 respectively.
3. The emotion electroencephalogram signal induction method based on conversation, as claimed in claim 1, wherein: the emotion marking is based on a human body model self-evaluation method, a VAD emotion dimensionality scoring table with text explanation is adopted to continuously and discretely mark emotion in voice, the text explanation is used as discrete emotion marking, and the continuous emotion marking and the discrete emotion marking are carried out in the electroencephalogram signal acquisition process.
4. The emotion electroencephalogram signal induction method based on conversation, as claimed in claim 3, wherein: and the value in the VAD emotion dimension scoring table is 1-9 points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110471023.0A CN113208635B (en) | 2021-04-29 | 2021-04-29 | Emotion electroencephalogram signal induction method based on conversation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110471023.0A CN113208635B (en) | 2021-04-29 | 2021-04-29 | Emotion electroencephalogram signal induction method based on conversation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113208635A true CN113208635A (en) | 2021-08-06 |
| CN113208635B CN113208635B (en) | 2022-05-20 |
Family
ID=77089916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110471023.0A Active CN113208635B (en) | 2021-04-29 | 2021-04-29 | Emotion electroencephalogram signal induction method based on conversation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113208635B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103690165A (en) * | 2013-12-12 | 2014-04-02 | 天津大学 | Cross-inducing-mode emotion electroencephalogram recognition and modeling method |
| CN106725452A (en) * | 2016-11-29 | 2017-05-31 | 太原理工大学 | Based on the EEG signal identification method that emotion induces |
| CN107361767A (en) * | 2017-08-04 | 2017-11-21 | 西南大学 | A kind of human emotion's potency classifying identification method using EEG signals |
| CN109117787A (en) * | 2018-08-10 | 2019-01-01 | 太原理工大学 | A kind of emotion EEG signal identification method and system |
| CN109199414A (en) * | 2018-10-30 | 2019-01-15 | 武汉理工大学 | A kind of audiovisual induction Emotion identification method and system based on EEG signals |
| CN112656427A (en) * | 2020-11-26 | 2021-04-16 | 山西大学 | Electroencephalogram emotion recognition method based on dimension model |
-
2021
- 2021-04-29 CN CN202110471023.0A patent/CN113208635B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103690165A (en) * | 2013-12-12 | 2014-04-02 | 天津大学 | Cross-inducing-mode emotion electroencephalogram recognition and modeling method |
| CN106725452A (en) * | 2016-11-29 | 2017-05-31 | 太原理工大学 | Based on the EEG signal identification method that emotion induces |
| CN107361767A (en) * | 2017-08-04 | 2017-11-21 | 西南大学 | A kind of human emotion's potency classifying identification method using EEG signals |
| CN109117787A (en) * | 2018-08-10 | 2019-01-01 | 太原理工大学 | A kind of emotion EEG signal identification method and system |
| CN109199414A (en) * | 2018-10-30 | 2019-01-15 | 武汉理工大学 | A kind of audiovisual induction Emotion identification method and system based on EEG signals |
| CN112656427A (en) * | 2020-11-26 | 2021-04-16 | 山西大学 | Electroencephalogram emotion recognition method based on dimension model |
Non-Patent Citations (3)
| Title |
|---|
| 孟辉: "一种基于情感诱发方法的情感语音数据库设计与实现", 《现代计算机》 * |
| 张琪;乔晓艳: "视听情绪诱发的脑电信号同步性分析", 《测试技术学报》 * |
| 环游号: "AI读懂说话人情绪，数据堂语音情感识别数据等你Pick", 《环游号》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113208635B (en) | 2022-05-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| De Linde et al. | The semiotics of subtitling | |
| Liu et al. | Recognizing vocal emotions in Mandarin Chinese: A validated database of Chinese vocal emotional stimuli | |
| Caucci et al. | Social and paralinguistic cues to sarcasm | |
| Calder | From sissy to sickening: The indexical landscape of/s/in SoMa, San Francisco | |
| Caro et al. | “Feeling” audio description: Exploring the impact of AD on emotional response | |
| Rilliard et al. | Multimodal indices to Japanese and French prosodically expressed social affects | |
| Purba et al. | Investigation of decoding fillers used in an English learning talk show “English with Alice” | |
| Di Giovanni | Audio description and reception-centred research | |
| CN113208635B (en) | Emotion electroencephalogram signal induction method based on conversation | |
| Tran et al. | “Kneweth one who makes these notes...”: Personality, individual differences, and liking of nouveau roman and existentialist literature and film. | |
| Milne et al. | Designing and evaluating interactive agents as social skills tutors for children with autism spectrum disorder | |
| Stadelmann | Language, cognition, interaction: Conceptual blending as discursive practice | |
| Gustafson-Capková | Emotions in speech: tagset and acoustic correlates | |
| Vaughan | Naturalistic emotional speech corpora with large scale emotional dimension ratings | |
| Endrass et al. | A data-driven approach to model Culture-specific Communication Management Styles for Virtual Agents | |
| Damiani | Art, design and neurodiversity | |
| Alhinti et al. | The Dysarthric expressed emotional database (DEED): An audio-visual database in British English | |
| Tu et al. | Establishment of Chinese Speech Emotion Database of Broadcasting | |
| Dehghan et al. | The effect of pedagogical films on the development of listening skill among Iranian EFL female and male learners | |
| Yanwei et al. | A Multimodal Discourse Analysis Of The Tourism Promotional Video “Xi’an China.” | |
| Mikhailovna | Crosscultural competence formation in the context of the dialogue of cultures | |
| Burczynska | Investigating the multimodal construal and reception of irony in film translation-an experimental approach | |
| Yanti et al. | VERBAL AND NONVERBAL ASPECTS IN REALIZING LANGUAGE POLITENESS IN THE FILM “SABTU BERSAMA BAPAK” MADE BY ADHITYA MULYA | |
| Tickle | Cross-language vocalisation of emotion: methodological issues | |
| Hafidzoh | The Relationship Between Students' Frequency of Watching English Movies and Their Listening Comprehension of Narrative |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |