JP2020187262A - Emotion estimation device, emotion estimation system, and emotion estimation method - Google Patents

Abstract

To accurately estimate emotions of a speaker.SOLUTION: The user device 1 includes: an acquisition unit 21 which acquires voice data VD indicating a sound including a voice of a speaker; a first estimation unit 252 which estimates whether emotion of the speaker belongs to a positive group GE1 to which positive emotions belong or a negative group GE2 to which negative emotions belong based on a recognition character string SD obtained by performing a voice recognition process on the voice data VD; a second estimation unit 253 which estimates whether the emotion of the speaker belongs to an excitement group GE3 to which excitement emotions belong or a non-excitement group GE4 to which non-excitement emotions belong based on a sound feature quantity of the voice data VD; and an emotion estimation unit 254 which estimates the speaker's emotion based on the estimation result of the first estimation unit 252 and the estimation result of the second estimation unit 253.SELECTED DRAWING: Figure 5

Description

The present invention relates to an emotion estimation device, an emotion estimation system, and an emotion estimation method.

In recent years, services for estimating emotions such as joy, anger, and sadness have become widespread. For example, in Patent Document 1, voice recognition processing is performed on voice data indicating a sound including the voice of the speaker, and based on the recognition character string obtained from the voice recognition processing, the speaker's voice is applied to each emotion. A score indicating the degree of possibility of being an emotion is calculated, a score of each emotion is calculated based on a sound feature amount indicated by voice data, and each emotion score obtained from a character string and a sound feature amount are used. It is disclosed that the emotion with the highest score among the average values of the obtained emotion scores is estimated as the speaker's emotion.

Japanese Unexamined Patent Publication No. 2012-73941

However, in the above-mentioned conventional technique, when the emotion is estimated based on the recognition character string obtained from the voice recognition process, the intonation of the voice is lost from the recognition character string, and the emotion estimation may be erroneous. On the other hand, when the emotion is estimated based on the sound feature, the sound feature does not include whether the content of the voice is positive or negative, and the emotion estimation may be erroneous. .. Therefore, the score of each emotion based on the recognition character string and the score of each emotion based on the sound feature amount may include a misestimated score. Therefore, an error may be included in the average value of the score of each emotion obtained from the character string and the score of each emotion obtained from the feature amount of the sound. As a result, the conventional emotion estimation method based on the average value may erroneously estimate the emotion.

The emotion estimation device according to the preferred embodiment of the present invention is based on an acquisition unit that acquires voice data indicating a sound including the voice of the speaker and a recognition character string obtained by performing voice recognition processing on the voice data. , The first estimation unit that estimates whether the speaker's emotion belongs to the first group to which the positive emotion belongs or the second group to which the negative emotion belongs, and the sound indicated by the voice data. Based on the feature quantity, the second estimation unit that estimates whether the speaker's emotion belongs to the third group to which the emotion during excitement belongs or the fourth group to which the emotion not during excitement belongs, and the above. It includes an emotion estimation unit that estimates the emotion of the speaker based on the estimation result of the first estimation unit and the estimation result of the second estimation unit.

The emotion estimation system according to a preferred embodiment of the present invention is an emotion estimation system including the above-mentioned emotion estimation device and a terminal device capable of communicating with the emotion estimation device, and the terminal device is a voice of the speaker. A sound collecting unit that collects sounds including the above, a transmitting unit that transmits the voice data indicating the sound including the voice of the speaker to the emotion estimation device, and the recognition character string from the emotion estimation device. The receiving unit that receives the emotion data indicating the emotion of the speaker estimated by the emotion estimation unit and the recognition character string are processed according to the emotion indicated by the emotion data, and the data obtained is output. It is provided with an output unit.

In the emotion estimation method according to a preferred embodiment of the present invention, the utterance is based on a recognition character string obtained by acquiring voice data indicating a sound including the voice of the speaker and performing voice recognition processing on the voice data. It is estimated whether the person's emotion belongs to the first group to which the positive emotion belongs or the second group to which the negative emotion belongs, and the utterance is based on the characteristic amount of the sound indicated by the voice data. It is estimated whether the emotions of the person belong to the third group to which the emotions at the time of excitement belong or the fourth group to which the emotions not at the time of excitement belong, and the emotions of the speaker belong to the first group and the second group. The emotion of the speaker is estimated based on the estimation result indicating which of the groups the speaker belongs to and the estimation result of which of the third group and the fourth group the speaker's emotion belongs to. The computer executes the processing to be performed.

According to the present invention, the emotion of the speaker can be estimated with high accuracy.

The block diagram which shows the user apparatus 1 which concerns on 1st Embodiment of this invention. The figure which shows the grouping of emotions. The figure which shows an example of the storage contents of the dictionary data 31 for analysis. The figure which shows an example of the memory content of the emotion classification data 33. The figure which shows the outline of the function of the user apparatus 1. The figure which shows the flowchart which shows the process of the estimation part 25. The block diagram which shows the user apparatus 1a which concerns on 2nd Embodiment. The figure which shows the outline of the function of the user apparatus 1a which concerns on 2nd Embodiment. The figure which shows the flowchart which shows the process of the estimation part 25a which concerns on 2nd Embodiment. The block diagram which shows the emotion estimation system 100.

1. 1. 1st Embodiment FIG. 1 is a block diagram showing a user apparatus 1 according to the first embodiment of the present invention. The user device 1 is assumed to be a smartphone. The user device 1 is an example of an “emotion estimation device”. However, any information processing device can be adopted as the user device 1, and for example, it may be a terminal-type information device such as a personal computer, or a portable type such as a notebook computer, a wearable terminal, or a tablet terminal. It may be an information terminal of.

The user device 1 is realized by a computer system including a processing device 2, a storage device 3, a display device 4, an operating device 5, a communication device 6, a sound emitting device 7, and a sound collecting device 8. Each element of the user device 1 is connected to each other by a single unit or a plurality of buses 9 for communicating information. The term "device" in the present specification may be read as another term such as a circuit, a device, or a unit. Further, each element of the user device 1 may be composed of a single device or a plurality of devices, and some elements of the user device 1 may be omitted. The sound collecting device 8 is an example of a “sound collecting unit”.

The user device 1 has a function of transmitting a recognition character string obtained by performing voice recognition processing on voice data indicating a sound including a voice of a speaker who is a user of the user device 1 to a device used by another person. Or, it has a function to read out the recognition character string and let others hear it. Further, the user device 1 estimates the emotion of the speaker based on the voice of the speaker, and adds a pictogram corresponding to the estimated emotion to the recognition character string to the recognition character string, or adds a pictogram corresponding to the estimated emotion to the recognition character string, or to the estimated emotion. By reading out the recognition character string with the corresponding intonation, it is possible to add emotional expressions necessary for communication. In order to facilitate communication, it is preferable to improve the estimation accuracy of the speaker's emotions.
In the first embodiment, the user device 1 has a result of grouping a plurality of emotions that a person can take according to whether they are positive or negative, and a result of grouping them according to whether they are excited or not. Estimate the speaker's emotions based on.

FIG. 2 is a diagram showing emotional grouping. When multiple emotions that a person can take are classified into a positive group GE1 to which positive emotions belong and a negative group GE2 to which negative emotions belong, joy belongs to positive group GE1 and anger and anger belong to negative group GE2. Sadness belongs. The positive group GE1 is an example of the "first group". Negative group GE2 is an example of a "second group".

Further, when a plurality of emotions that a person can take are classified into an excitement group GE3 to which emotions at the time of excitement belong and a non-excitement group GE4 to which emotions at the time of non-excitement belong, the excitement group GE3 belongs to joy and anger and is excited. Sadness belongs to the non-excited group GE4 to which non-time emotions belong. The excitement group GE3 is an example of the "third group". The non-excited group GE4 is an example of the "fourth group".

The explanation is returned to FIG. The processing device 2 is a processor that controls the entire user device 1, and is composed of, for example, a single chip or a plurality of chips. The processing device 2 is composed of, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, registers, and the like. Part or all of the functions of the processing device 2 are realized by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), etc. You may. The processing device 2 executes various processes in parallel or sequentially.

The storage device 3 is a recording medium that can be read by the processing device 2, and stores a plurality of programs including the control program PR executed by the processing device 2, the analysis dictionary data 31, and the emotion classification data 33. The storage device 3 is composed of, for example, one or more types of storage circuits such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and RAM (Random Access Memory).

FIG. 3 is a diagram showing an example of the stored contents of the analysis dictionary data 31. The analysis dictionary data 31 is data in which part of speech, part of speech subclassification, and original form data are associated with each other for each morpheme. A morpheme is a character string that is the smallest unit of a meaningful expression element. Part of speech is a type of word classified according to its grammatical nature, such as nouns, verbs, and adjectives. Part of speech subclassification is an item in which part of speech is further subdivided. The original form data is a character string indicating the original form of the word when the word is utilized by the corresponding morpheme, and is the same character string as the corresponding morpheme when the word is not utilized by the relevant morpheme.

FIG. 4 is a diagram showing an example of the stored contents of the emotion classification data 33. The emotion classification data 33 is data in which a character string is classified into any of joy, anger, and sadness. In the example of FIG. 4, the character string group 331 classified as joy includes "happy", "pass", "win", "win", and the like. Similarly, the character string group 332 classified as anger includes "irritated", "mucked up" and the like. Similarly, the character string group 333 classified as sadness includes "sad", "losing", and the like.

The explanation is returned to FIG. The display device 4 displays various images under the control of the processing device 2. For example, various display panels such as a liquid crystal display panel or an organic EL (Electro Luminescence) display panel are preferably used as the display device 4.

The operation device 5 is a device for inputting information used by the user device 1. The operation device 5 accepts an operation by the user. Specifically, the operating device 5 accepts an operation for inputting a code such as a number and a character and an operation for selecting an icon displayed by the display device 4. For example, a touch panel that detects contact with the display surface of the display device 4 is suitable as the operation device 5. The operation device 5 may include an operator that can be operated by the user. The operator is, for example, a stylus.

The communication device 6 is hardware (transmission / reception device) for communicating with another device via a network. The communication device 6 is also called, for example, a network device, a network controller, a network card, a communication module, or the like.

The sound emitting device 7 is composed of, for example, a speaker, and emits sound under the control of the processing device 2. The sound collecting device 8 is composed of, for example, a microphone and an AD converter, and collects sound including the voice of the speaker under the control of the processing device 2. The microphone converts the collected voice into an electric signal. The AD converter AD-converts the electric signal converted by the microphone and converts it into the voice data VD shown in FIG. The sound indicated by the voice data VD may include noise emitted from the surroundings of the speaker in addition to the voice of the speaker.

1.1. The function processing device 2 of the first embodiment functions as an acquisition unit 21, an estimation unit 25, and an output unit 26 by reading and executing the control program PR from the storage device 3.
The function realized by the processing apparatus 2 will be described with reference to FIG.

FIG. 5 is a diagram showing an outline of the functions of the user device 1. The acquisition unit 21 acquires voice data VD indicating the voice of the speaker collected by the sound collecting device 8. The estimation unit 25 estimates the emotion of the speaker based on the voice data VD. Specifically, the estimation unit 25 includes a voice recognition processing unit 251, a first estimation unit 252, a second estimation unit 253, and an emotion estimation unit 254.

The voice recognition processing unit 251 performs voice recognition processing on the voice data VD and outputs the recognition character string SD. The voice recognition processing unit 251 outputs the recognition character string SD by various methods including a method of recognizing a character string from a voice by using, for example, an acoustic model and a language model prepared in advance.

The first estimation unit 252 executes the character string emotion estimation process. The character string emotion estimation process estimates whether the speaker's emotion belongs to the positive group GE1 or the negative group GE2 based on the recognition character string SD.

More specifically, the first estimation unit 252 includes a morphological analysis processing unit 2521 and an emotion score calculation processing unit 2522. The morphological analysis processing unit 2521 refers to the analysis dictionary data 31 to perform morphological analysis processing on the recognition character string SD, and outputs the corrected recognition character string CSD. The morphological analysis process is a process of decomposing the recognition character string SD into morphemes. In the morphological analysis process, the part of speech and the part of speech subclassification of the analysis dictionary data 31 are used. The corrected recognition character string CSD is a character string excluding a character string that is unnecessary for estimating the emotion of the speaker, such as an interjection.

The emotion score calculation processing unit 2522 indicates the degree of possibility that the emotion is the speaker's emotion for each emotion by comparing the character string included in the emotion classification data 33 with the corrected recognition character string CSD. Calculate the score. More specifically, the emotion score calculation processing unit 2522 corresponds to the character string included in the corrected recognition character string CSD when the corrected recognition character string CSD includes the character string included in the emotion classification data 33. Increases emotional scores.
For example, if the corrected recognition character string CSD is "winning the game today", the emotion score calculation processing unit 2522 outputs the following score for each emotion.

Joy 1
Anger 0
Sadness 0
In the above example, since the corrected recognition character string CSD includes the “win” included in the emotion classification data 33, the emotion score calculation processing unit 2522 increases the joy score corresponding to the “win” by 1. Let me. The amount of the score to be increased is not limited to 1, and may differ for each character string included in the emotion classification data 33. For example, the amount of increase in the score of the character string that indicates more joy may be set to 2. Further, when the corrected recognition character string CSD includes the character string included in the emotion classification data 33 and the character string that emphasizes the content, the emotion score calculation processing unit 2522 increases the amount of increase in the emotion score. You may. For example, if the corrected recognition character string CSD is "very happy to win the game today", the corrected recognition character string CSD contains "happy" included in the emotion classification data 33 and is "very". Since the character string emphasizing the content is included, the emotion score calculation processing unit 2522 increases, for example, the joy score by 2. Which of the corrected recognition character string CSDs is a character string that emphasizes the content can be determined by the morpheme obtained by the morphological analysis process. In the following example, for ease of explanation, the amount of score to be increased is assumed to be 1.
Further, when the corrected recognition character string CSD includes a character string included in the emotion classification data 33 and a character string denying the content, the emotion score calculation processing unit 2522 is included in the corrected recognition character string CSD. A process different from the process of increasing the emotional score corresponding to the character string may be executed. For example, if the corrected recognition character string CSD is "could not win the game today", the corrected recognition character string CSD includes "win" included in the emotion classification data 33, but is said to be "not". Since the character string denying the content is included, the emotion score calculation processing unit 2522 increases the score of sadness by 1, for example. Which character string of the corrected recognition character string CSD is a character string whose content is negated can be determined by the morpheme obtained by the morphological analysis process. In this way, it is possible to estimate whether the corrected recognition character string CSD has positive content or negative content by the morphological analysis process. In the following example, if the corrected recognition character string CSD includes the character string included in the emotion classification data 33, the emotion corresponding to the character string included in the corrected recognition character string CSD is used for ease of explanation. It will be explained as increasing the score of.

The first estimation unit 252 outputs the first emotion group data GD1 indicating whether the speaker's emotion belongs to the positive group GE1 or the negative group GE2 based on the score for each emotion. The first emotion group data GD1 has, for example, the following two aspects.

The first aspect of the first emotion group data GD1 is either an identifier indicating a positive group GE1 or an identifier indicating a negative group GE2. For example, the first estimation unit 252 outputs an identifier indicating the positive group GE1 as the first emotion group data GD1 when the following equation (1) is satisfied. On the other hand, when the equation (1) is not satisfied, the first estimation unit 252 outputs an identifier indicating the negative group GE2 as the first emotion group data GD1.

Joy score> α × (anger score + sadness score) / 2 (1)

α is a value set by, for example, the developer of the user device 1 or the speaker.

The second aspect of the first emotion group data GD1 includes either an identifier indicating a positive group GE1 or an identifier indicating a negative group GE2, and a score of each emotion.

The second estimation unit 253 executes the voice emotion estimation process. In the voice emotion estimation process, the speaker's emotion is either the excitement group GE3 to which the emotion at the time of excitement belongs or the non-excitement group GE4 to which the emotion at the time of excitement belongs based on the feature amount of the sound indicated by the voice data VD. It is a process of estimating whether or not it belongs to.

More specifically, the second estimation unit 253 has a sound feature amount extraction processing unit 2531 and a learning model execution processing unit 2532. The sound feature amount extraction processing unit 2531 extracts the sound feature amount from the voice data VD. The sound feature amount is a feature amount indicating the sound feature indicated by the voice data VD. Sound features include, for example, MFCC (Mel-Frequency Cepstrum Coefficients) 12 dimensions, loudness, fundamental frequency (F0), voice probability, zero crossover ratio, HNR (Harmonics-to-Noise-Ratio), and their first derivative. , MFCC and loudness quadratic differentiation total 47. Loudness is the loudness of a sound, which indicates the intensity of the sound felt by human hearing. The voice probability indicates the probability that the sound indicated by the voice data VD includes voice. The zero crossing rate is the number of times the sound pressure becomes zero. At the time of excitement, for example, the fundamental frequency tends to be higher and the loudness tends to be larger than at the time of non-excitation. Further, the sound feature amount extraction processing unit 2531 may execute a correction process on the voice data VD and extract the sound feature amount from the corrected voice data obtained by executing the correction process. The correction process is, for example, one or both of a process of removing silent portion of data from the voice data VD and a process of removing noise contained in the sound indicated by the voice data VD.
The learning model execution processing unit 2532 inputs the feature amount of the extracted sound into the learning model learned in advance, and outputs the second emotion group data GD2 based on the output result obtained from the learning model. The second emotion group data GD2 indicates whether the speaker's emotion belongs to the excited group GE3 or the non-excited group GE4.
The learning model learned in advance is, for example, a model that outputs a score for each emotion when a sound feature amount is input. The second estimation unit 253 outputs the second emotion group data GD2 indicating that the speaker's emotion belongs to the excitement group GE3 when the following equation (2) is satisfied. On the other hand, when the equation (2) is not satisfied, the second estimation unit 253 outputs the second emotion group data GD2 indicating that the speaker's emotion belongs to the non-excited group GE4.

(Score of joy + score of anger) / 2> β x score of sadness (2)

β is, for example, a value set by the developer or speaker of the user device 1.

The second emotion group data GD2 has, for example, the following two aspects. The first aspect of the second emotion group data GD2 is either an identifier indicating the excitement group GE3 or an identifier indicating the non-excitement group GE4. The second aspect of the second emotion group data GD2 is either one of the identifier indicating the excitement group GE3 and the identifier indicating the non-excitement group GE4, and the score of each emotion output by the learning model learned in advance.

The emotion estimation unit 254 estimates the emotion of the speaker based on the estimation result indicated by the first emotion group data GD1 and the estimation result indicated by the second emotion group data GD2.

More specifically, when the first emotion group data GD1 indicates that the speaker's emotion belongs to the positive group GE1, the emotion estimation unit 254 estimates that the speaker's emotion is joy.
Further, when the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2 and the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3, the emotion estimation unit 254 estimates that the speaker's emotions are anger.
When the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2 and the second emotion group data GD2 indicates that the speaker's emotion belongs to the non-excited group GE4, the emotion estimation unit 254 Presumes that the speaker's emotions are sad.
The emotion estimation unit 254 outputs emotion data ED indicating the estimated emotion of the speaker. The emotional data ED has, for example, the following two aspects. The first aspect of the emotion data ED is an identifier indicating the estimated emotion of the speaker. The identifier indicating emotion includes an identifier indicating joy, an identifier indicating anger, and an identifier indicating sadness. A second aspect of the emotion data ED is an identifier indicating the estimated speaker's emotion and an estimated speaker's emotion score. The estimated speaker emotion score is included in, for example, the estimated speaker emotion score included in the second aspect of the first emotion group data GD1 and the second aspect of the second emotion group data GD2. It is the total value or the average value with the estimated speaker's emotional score.

The output unit 26 outputs the data obtained by performing processing according to the emotion indicated by the emotion data ED on the recognition character string SD obtained by the voice recognition processing unit 251. There are two modes of processing according to emotions, for example, as shown below.
The first aspect of the process according to the emotion is a process of adding a figure embodying the emotion to the recognition character string SD. The figures that embody emotions are, for example, pictograms that embody emotions and emoticons that embody emotions. A pictogram is an image associated with a character code. The character code is, for example, Unicode. An emoticon is a character string that expresses a face by combining symbols and characters. In the following description, a figure that embodies emotions will be described as a pictogram that embodies emotions. The pictogram that embodies joy is, for example, a pictogram that shows a smile. The pictogram that embodies anger is, for example, a pictogram that shows the face of anger. The pictogram that embodies sadness is, for example, a pictogram that shows a crying face. Further, when the emotion data ED is the second aspect, the output unit 26 outputs a pictogram embodying the emotion indicated by the emotion data ED and having a depth corresponding to the score included in the emotion data ED. , May be determined as a pictogram to be added to the recognition character string SD. For example, when the emotion indicated by the emotion data ED is sadness and the score included in the emotion data ED is equal to or less than a predetermined threshold value, the output unit 26 adds a pictogram indicating a face spilling tears to the recognition character string SD. Decide as a pictogram to do. On the other hand, when the emotion indicated by the emotion data ED is sad and the score included in the emotion data ED is larger than a predetermined threshold value, the output unit 26 adds a pictogram indicating a crying face to the recognition character string SD. To determine as. The emoji showing a crying face embodies deeper sadness compared to the emoji showing a tearful face.
The output unit 26 outputs a character string with a pictogram obtained by adding a pictogram to the recognition character string SD. For example, there are two positions for adding pictograms as shown below. The first position is the end of the recognition character string SD. The second position is next to the character string included in the emotion classification data 33 in the recognition character string SD. The display device 4 displays an image based on the character string with pictograms output by the output unit 26.

The second aspect of the emotion-based process is a process of generating a synthetic voice read aloud by adding emotion-based intonation. Inflection is to increase or decrease the reading speed, or to increase or decrease the volume. Pleasure-based intonation is, for example, speeding up reading. Anger-based intonation is, for example, increasing the volume. Sadness-based intonation is, for example, reducing the volume. The output unit 26 outputs data indicating a synthetic voice read aloud with an emotion-based intonation added. Then, the output unit 26 adds emotion-based intonation to the synthetic voice indicated by the generated data, adds emotion-based intonation, and outputs data indicating the synthetic voice read aloud. The sound emitting device 7 emits a synthetic voice indicated by the data output by the output unit 26.

1.2. Operation of First Embodiment Next, the process executed by the estimation unit 25 will be described with reference to FIG.

FIG. 6 is a flowchart showing the processing of the estimation unit 25. The processes of step S3, step S4, step S6, step S7, and step S8 shown in FIG. 6 correspond to the emotion estimation unit 254. The voice recognition processing unit 251 performs voice recognition processing on the voice data VD to obtain the recognition character string SD (step S1). Next, the first estimation unit 252 executes the character string emotion estimation process on the recognition character string SD and outputs the first emotion group data GD1 (step S2).

The emotion estimation unit 254 determines whether or not the first emotion group data GD1 indicates the positive group GE1 (step S3). In other words, in step S3, the emotion estimation unit 254 determines whether the first emotion group data GD1 indicates a positive group GE1 or a negative group GE2. When the first emotion group data GD1 indicates the positive group GE1 and the determination result in step S3 is affirmative, the emotion estimation unit 254 estimates that the speaker's emotion is joy (step S4).

When the first emotion group data GD1 indicates the negative group GE2 and the determination result in step S3 is negative, the second estimation unit 253 executes the voice emotion estimation process on the voice data VD, and the second emotion group data. Output GD2 (step S5). The emotion estimation unit 254 determines whether or not the second emotion group data GD2 indicates the excitement group GE3. In other words, in step S5, the emotion estimation unit 254 determines whether the second emotion group data GD2 indicates the excited group GE3 or the non-excited group GE4 (step S6).

When the first emotion group data GD1 indicates the negative group GE2 and the second emotion group data GD2 indicates the excitement group GE3, the emotion estimation unit 254 estimates that the speaker's emotion is anger (step S7). .. When the first emotion group data GD1 indicates the negative group GE2 and the second emotion group data GD2 indicates the non-excited group GE4, the emotion estimation unit 254 estimates that the speaker's emotion is sad (step S8). ).

After the processing of step S4, step S7, or step S8 is completed, the estimation unit 25 ends a series of processing shown in FIG.

1.3. Effect of First Embodiment As described above, according to the first embodiment, the user device 1 estimates the estimation result of the character string emotion estimation process for the recognition character string SD and the estimation of the voice emotion estimation process for the voice data VD. Estimate the speaker's emotions based on the results. Since the character string emotion estimation process focuses on the meaning and content of the recognition character string SD, it is possible to determine with high accuracy whether the recognition character string SD based on the speaker's voice has positive content or negative content. On the other hand, the intonation of the voice clearly shows whether or not the speaker is excited. Since the recognition character string SD is merely a character string, the intonation of the voice is lost. Speakers' emotions include joy and anger that appear when they are excited, and sadness that appears when they are not excited. Therefore, if it is attempted to estimate whether the speaker's emotion is an excited emotion or a non-excited emotion from the recognition character string SD, an erroneous estimation may occur. For example, in the emotion score calculation processing unit 2522, the sadness score may be calculated to be the highest based on the recognition character string SD even though the true emotion of the speaker is anger. Further, in the emotion score calculation processing unit 2522, the anger score may be calculated to be the highest based on the recognition character string SD even though the true emotion of the speaker is sadness. That is, when trying to distinguish between emotions during excitement and emotions during non-excitement based on the recognition character string SD, anger and sadness may be confused. However, since the first estimation unit 252, which estimates emotions based on the recognition character string SD, estimates anger and sadness, which may be confused, as one group, it is possible to eliminate erroneous estimation due to confusion between anger and sadness. Can be done.
Regarding the voice emotion estimation process, among the sound features, there are features such as fundamental frequency and loudness that tend to have values that differ greatly between when excited and when not excited. Therefore, in the voice emotion estimation process, it is possible to accurately estimate whether the emotion of the speaker is an emotion during excitement or an emotion during non-excitement. On the other hand, the sound features do not include the meaning and content of the speaker's utterance. Speakers' emotions include joy that appears in positive times and anger and sadness that appear in negative times. Therefore, if it is attempted to estimate whether the speaker's emotion is a positive emotion or a negative emotion from the feature amount of the sound, an erroneous estimation may occur. For example, in the learning model in the learning model execution processing unit 2532, the anger score may be calculated to be the highest even though the true emotion of the speaker is joy. Further, in the learning model in the learning model execution processing unit 2532, the joy score may be calculated to be the highest even though the true emotion of the speaker is anger. That is, trying to distinguish between positive and negative emotions based on sound features can confuse joy with anger. However, the second estimation unit 253, which estimates emotions based on sound features, estimates joy and anger, which may be confused, as one group, so that erroneous estimation due to confusion between joy and anger is eliminated. Can be done.
As described above, according to the first embodiment, emotions that may be confused are estimated as one group, so that erroneous estimation can be suppressed. For example, the speaker's emotion is estimated more accurately than the case where the speaker's emotion is estimated by the average value of the score of each emotion based on the recognition character string SD and the score of each emotion based on the sound features. Becomes possible.

Further, according to the first embodiment, when the first emotion group data GD1 indicates that the speaker's emotion belongs to the positive group GE1, the emotion estimation unit 254 estimates that the speaker's emotion is joy. When the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2, and the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3, the emotion estimation unit 254 , Estimate that the speaker's emotions are anger. When the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2, and the second emotion group data GD2 indicates that the speaker's emotion belongs to the non-excited group GE4, the emotion estimation unit 254 Presumes that the speaker's emotions are sad.
As described above, when the first emotion group data GD1 indicates that the speaker's emotion belongs to the positive group GE1, the emotion estimation unit 254 estimates the speaker's emotion without referring to the second emotion group data GD2. Will be possible. Further, even when the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2, the emotion estimation unit 254 refers to the second emotion group data GD2 to refer to the speaker's emotion. Can be estimated accurately.

2. 2. 2nd Embodiment In the 1st embodiment, the estimation unit 25 executes the character string emotion estimation process by the 1st estimation unit 252, and when the 1st emotion group data GD1 indicates the negative group GE2, the 2nd estimation unit 253 Execute voice emotion estimation processing. On the other hand, in the second embodiment, the estimation unit 25a executes the voice emotion estimation process by the second estimation unit 253, and when the second emotion group data GD2 indicates the excitement group GE3, the character string emotion by the first estimation unit 252. Perform the estimation process. Hereinafter, the user device 1a according to the second embodiment will be described. Regarding the elements whose actions or functions are the same as those of the first embodiment in the second embodiment illustrated below, the reference numerals are used in the above description, and detailed description of each is appropriately omitted.

2.1. Function of the second embodiment FIG. 7 is a block diagram showing a user device 1a according to the second embodiment. The user device 1a is realized by a computer system including a processing device 2a, a storage device 3a, a display device 4, an operating device 5, a communication device 6, a sound emitting device 7, and a sound collecting device 8. The storage device 3a is a recording medium that can be read by the processing device 2a, and stores a plurality of programs including the control program PRa executed by the processing device 2a.

The processing device 2a functions as an acquisition unit 21, an estimation unit 25a, and an output unit 26 by reading and executing the control program PRa from the storage device 3a.

FIG. 8 is a diagram showing an outline of the function of the user device 1a according to the second embodiment. The estimation unit 25a includes a voice recognition processing unit 251, a first estimation unit 252, a second estimation unit 253, and an emotion estimation unit 254a.

When the second emotion group data GD2 indicates that the speaker's emotion belongs to the non-excited group GE4, the emotion estimation unit 254a estimates that the speaker's emotion is sad.
Further, when the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3 and the first emotion group data GD1 indicates that the speaker's emotion belongs to the positive group GE1, the emotion estimation unit 254a presumes that the speaker's emotions are joy.
When the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3 and the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2, the emotion estimation unit 254a , Estimate that the speaker's emotions are anger.

2.2. Operation of the Second Embodiment Next, the process executed by the estimation unit 25a will be described with reference to FIG.

FIG. 9 is a flowchart showing the processing of the estimation unit 25a according to the second embodiment. The processes of step S22, step S23, step S26, step S27, and step S28 shown in FIG. 9 correspond to the emotion estimation unit 254a. The second estimation unit 253 executes the voice emotion estimation process on the voice data VD and outputs the second emotion group data GD2 (step S21).

The emotion estimation unit 254a determines whether or not the second emotion group data GD2 indicates the non-excited group GE4 (step S22). In other words, in step S22, the emotion estimation unit 254a determines whether the second emotion group data GD2 indicates the excited group GE3 or the non-excited group GE4. When the second emotion group data GD2 indicates the non-excited group GE4 and the determination result in step S22 is affirmative, the emotion estimation unit 254a estimates that the speaker's emotion is sad (step S23).

On the other hand, when the second emotion group data GD2 indicates the excitement group GE3 and the determination result in step S22 is negative, the voice recognition processing unit 251 performs voice recognition processing on the voice data VD and recognizes the recognition character string SD. (Step S24). Next, the first estimation unit 252 executes the character string emotion estimation process on the recognition character string SD and outputs the first emotion group data GD1 (step S25).

The emotion estimation unit 254a determines whether or not the first emotion group data GD1 indicates a positive group GE1 (step S26). In other words, in step S26, the emotion estimation unit 254a determines whether the first emotion group data GD1 indicates a positive group GE1 or a negative group GE2.

When the second emotion group data GD2 indicates the excitement group GE3 and the first emotion group data GD1 indicates the positive group GE1, the emotion estimation unit 254a estimates that the speaker's emotion is joy (step S27). .. On the other hand, when the second emotion group data GD2 shows the excitement group GE3 and the first emotion group data GD1 shows the negative group GE2, the emotion estimation unit 254a estimates that the speaker's emotion is anger (step). S28).

After the processing of step S23, step S27, or step S28 is completed, the estimation unit 25a ends a series of processing shown in FIG.

2.3. Effect of Second Embodiment As described above, according to the second embodiment, when the second emotion group data GD2 indicates that the speaker's emotion belongs to the non-excited group GE4, the emotion estimation unit 254a speaks. It is presumed that one's feelings are sadness. Further, when the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3 and the first emotion group data GD1 indicates that the speaker's emotion belongs to the positive group GE1, the emotion estimation unit 254a presumes that the speaker's emotions are joy. When the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3 and the first emotion group data GD1 indicates that the speaker's emotion belongs to the negative group GE2, the emotion estimation unit 254a , Estimate that the speaker's emotions are anger.
According to the above, when the second emotion group data GD2 indicates that the speaker's emotion belongs to the non-excited group GE4, the emotion estimation unit 254a refers to the first emotion group data GD1. It becomes possible to estimate the emotion of the speaker. Further, even when the second emotion group data GD2 indicates that the speaker's emotion belongs to the excitement group GE3, the speaker's emotion can be estimated accurately by referring to the first emotion group data GD1. Becomes possible.

3. 3. Modifications The present invention is not limited to the embodiments exemplified above. A specific mode of modification is illustrated below. Two or more aspects arbitrarily selected from the following examples may be merged.

(1) In each of the above-described forms, in the equation (1), the first estimation unit 252 has a “joy score” on the left side and an “α × (anger score + sadness score) / 2” on the right side. When the absolute value of the difference from is equal to or greater than a predetermined value, the first emotion group data GD1 indicating that the speaker's emotion belongs to either the positive group GE1 or the negative group GE2 is output, and the absolute value of the above difference is absolute. When the value is less than a predetermined value, the first emotion group data GD1 indicating that the speaker's emotion is unknown may be output.
Similarly, in the second estimation unit 253, in equation (2), the absolute value of the difference between the left side "(joy score + anger score) / 2" and the right side "β x sadness score" Is equal to or greater than a predetermined value, the second emotion group data GD2 indicating that the speaker's emotion belongs to either the excited group GE3 or the non-excited group GE4 is output, and the absolute value of the above difference is less than the predetermined value. In some cases, the second emotion group data GD2 indicating that the speaker's emotion is unknown may be output.
When either one of the first emotion group data GD1 and the second emotion group data GD2 indicates that the inventor's emotion is unknown, the emotion estimation unit 254 of the speaker is based on the other emotion group data. Emotions may be estimated.
Here, it is assumed that the first emotion group data GD1 and the second emotion group data GD2 are the second aspect. For example, when the first emotion group data GD1 indicates that the speaker's emotion is unknown, the emotion estimation unit 254 has the highest score among the scores of each emotion included in the second emotion group data GD2. Estimate the emotion as the speaker's emotion. Similarly, when the second emotion group data GD2 indicates that the speaker's emotion is unknown, the emotion estimation unit 254 determines the highest score among the scores of each emotion included in the first emotion group data GD1. Estimate the emotions that the speaker has.

(2) In the second embodiment, in step S25, the first estimation unit 252 recognizes the character string group 331 classified as joy and the character string group 332 classified as anger, which are included in the emotion classification data 33. By comparing with the character string SD, it may be estimated whether the speaker's emotion belongs to the positive group GE1 or the negative group GE2. In other words, the first estimation unit 252 does not have to compare the character string group 333 classified as sadness included in the emotion classification data 33 with the recognition character string SD. The first estimation unit 252 can shorten the time required for the character string emotion estimation process because the character string group 333 classified into sadness included in the emotion classification data 33 is not compared with the recognition character string SD.

(3) In the first embodiment, in step S3, when the first emotion group data GD1 indicates the positive group GE1, the second estimation unit 253 did not execute the voice emotion estimation process, but it may be executed. Similarly, in the second embodiment, in step S22, when the second emotion group data GD2 indicates the non-excited group GE4, the first estimation unit 252 did not execute the character string emotion estimation process, but even if it does. Good. When the first emotion group data GD1 indicates a positive group GE1 and the second emotion group data GD2 indicates a non-excited group GE4, the emotion estimation unit 254 may determine that the speaker's emotion cannot be estimated.
In this way, when the estimation result of the first estimation unit 252 and the estimation result of the second estimation unit 253 are inconsistent, it is shown that one of the estimation results is an erroneous estimation. The unit 254 can suppress the output of an erroneous estimation result.

(4) As described above, when the first emotion group data GD1 indicates the positive group GE1 and the second emotion group data GD2 indicates the non-excited group GE4, the emotion estimation unit 254 estimates the emotion of the speaker. You may decide that it is impossible. When the emotion estimation unit 254 determines that the speaker's emotion cannot be estimated, the output unit 26 outputs at least one graphic character string out of the two pictogram character strings shown below. The first character string with pictograms is a character string with pictograms obtained by applying a process of adding pictograms embodying joy to the recognition character string SD. The second character string with a pictogram is a character string obtained by subjecting the recognition character string SD to a process in which a pictogram embodying sadness is added. The output unit 26 may output both the first character string with pictograms and the second character string with pictograms, or may output either one.
For example, assume that the first emotion group data GD1 and the second emotion group data GD2 are the second aspect. The output unit 26 outputs a character string with a first pictogram when the joy score included in the first emotion group data GD1 is equal to or higher than a predetermined value, and the sadness score included in the second emotion group data GD2 is predetermined. If it is greater than or equal to the value, a second character string with a pictogram is output.
The speaker, who is a user, looks at the character string with a pictogram displayed on the display device 4 and operates the operation device 5 to select a character string with a pictogram that is close to his / her emotion.
As described above, even if the user device 1 determines that the emotion of the speaker cannot be estimated, the user device 1 can select an appropriate emotion close to the emotion of the speaker by letting the speaker select the emotion.

(5) A service that supports communication with another person may be provided by the emotion estimation system 100 including the user device 1c and the user device 1c and the accessible server device 101.

FIG. 10 is a block diagram showing an emotion estimation system 100. The emotion estimation system 100 includes a user device 1c and a server device 101. In this modification, the server device 101 is an example of the “emotion estimation device”. The user device 1c is an example of a "terminal device".

The user device 1c is realized by a computer system including a processing device 2c, a storage device 3c, a display device 4, an operating device 5, a communication device 6, a sound emitting device 7, and a sound collecting device 8. The storage device 3c is a recording medium that can be read by the processing device 2c, and stores a plurality of programs including the control program PRc executed by the processing device 2c. The communication device 6 accesses the server device 101 via the network.

The processing device 2c functions as a transmitting unit 22, a receiving unit 23, and an output unit 26 by reading and executing the control program PRc from the storage device 3c.

The transmission unit 22 transmits the voice data VD obtained by the sound collecting device 8 to the server device 101. The receiving unit 23 receives the recognition character string SD and the emotion data ED from the server device 101.

The server device 101 is realized by a computer system including a processing device 2C, a storage device 3C, and a communication device 6C. Each element of the server device 101 is connected to each other by a single unit or a plurality of buses 9C for communicating information. The storage device 3C is a recording medium that can be read by the processing device 2C, and stores a plurality of programs including the control program PRC executed by the processing device 2C, analysis dictionary data 31, and emotion classification data 33. The communication device 6C accesses the user device 1c via the network.

The processing device 2C functions as an acquisition unit 21C and an estimation unit 25 by reading and executing the control program PRC from the storage device 3C.

The acquisition unit 21C acquires the voice data VD from the user device 1c. The estimation unit 25 estimates the emotion of the speaker based on the voice data VD, and transmits the emotion data ED indicating the estimated emotion and the recognition character string SD to the user device 1c.

According to this modification, since the server device 101 estimates the emotion of the speaker, it is possible to suppress the load applied to the user device 1c as compared with the user device 1 in the first embodiment.
In this modification, the processing device 2c functions as the output unit 26, in other words, the recognition character string SD is processed according to the emotion indicated by the emotion data ED, but the processing device 2C is the output unit 26. May function as. When the processing device 2C functions as the output unit 26, the server device 101 executes a process corresponding to the emotion indicated by the emotion data ED on the recognition character string SD, and the data obtained by this process is used as the user device 1c. Send to.

(6) In each of the above-described aspects, the estimation unit 25 may execute the first estimation unit 252 and the second estimation unit 253 in parallel.

(7) In each of the above-described aspects, the user device 1 does not have to have the sound collecting device 8. When the sound collecting device 8 is not provided, the user device 1 may acquire the voice data VD via the communication device 6 or may acquire the voice data VD stored in the storage device 3.

(8) In each of the above aspects, the user device 1 does not have to have the sound emitting device 7.

(9) In each of the above aspects, the user device 1 may be a smart speaker. When the user device 1 is a smart speaker, the user device 1 does not have to have the display device 4 and the operation device 5.

(10) In each of the above-described aspects, the emotion classification data 33 rejoices, angers, and makes each of the plurality of morphemes utilized by a certain word, such as "win" and "win", as shown in FIG. , But not limited to this. For example, the emotion classification data 33 may classify the character string registered in the prototype data of the analysis dictionary data 31 into any of joy, anger, and sadness. For example, the emotion classification data 33 classifies the character strings “happy”, “pass”, and “win” registered in the prototype data of the analysis dictionary data 31 into joy. The emotion score calculation processing unit 2522 decomposes the corrected recognition character string CSD for each morpheme, and converts the decomposed morpheme into a character string registered in the original form data of the analysis dictionary data 31. Then, the emotion score calculation processing unit 2522 corresponds to the character string included in the corrected recognition character string CSD when the character string obtained by the conversion matches the character string included in the emotion classification data 33. Increase your emotional score.

(11) In each of the above aspects, the emotion score calculation processing unit 2522 calculated the score for each emotion with respect to the corrected recognition character string CSD, but calculated the score for each emotion with respect to the recognition character string SD. You may. However, the recognition character string SD includes a character string that is unnecessary for estimating emotions. Therefore, by calculating the score for each emotion for the corrected recognition character string CSD, it is possible to improve the estimation accuracy of the emotion as compared with the case of calculating the score for each emotion for the recognition character string SD. It will be possible.

(12) In each of the above aspects, an example in which the speaker speaks Japanese is used, but each of the above aspects can be applied regardless of the language spoken by the speaker. For example, even when the speaker speaks English, French, Chinese, or the like other than Japanese, each of the above aspects can be applied. For example, when the speaker speaks English, the analysis dictionary data 31 is data related to English morphemes, and the emotion classification data 33 is data that classifies English words into any of joy, anger, and sadness. Good.

(13) In each of the above aspects, in the pre-learned learning model in the learning model execution processing unit 2532, when the sound feature amount is input, the speaker's emotion is either the excited group GE3 or the non-excited group GE4. It may be a model that outputs the second emotion group data GD2 indicating whether or not it belongs to.

(14) In each of the above aspects, emotions other than joy, anger, and sadness may be applied according to the grouping of emotions. For example, healing may belong to the positive group GE1 and the non-excited group GE4.

(15) The block diagram used in the description of each of the above-described embodiments shows a block of functional units. These functional blocks (components) are realized by any combination of hardware and / or software. Further, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one physically and / or logically coupled device, or directly and / or indirectly by two or more physically and / or logically separated devices. (For example, wired and / or wireless) may be connected and realized by these plurality of devices.

(16) The order of the processing procedures, sequences, flowcharts, etc. in each of the above-described aspects may be changed as long as there is no contradiction. For example, the methods described herein present elements of various steps in an exemplary order, and are not limited to the particular order presented.

(17) In each of the above-described aspects, the input / output information and the like may be stored in a specific place (for example, a memory) or may be managed by a management table. Input / output information and the like can be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

(18) In each of the above-described aspects, the determination may be made by a value represented by 1 bit (0 or 1) or by a boolean value (Boolean: true or false). , May be done by numerical comparison (eg, comparison with a given value).

(19) In each of the above-described aspects, a portable information processing device such as a smartphone is illustrated as the user device 1, but the specific form of the user device 1 is arbitrary and is not limited to the above-mentioned examples of each form. .. For example, a portable or stationary personal computer may be used as the user device 1.

(20) In each of the above-described aspects, the storage device 3 is a recording medium that can be read by the processing device 2, and examples thereof include a ROM and a RAM. However, a flexible disk, a magneto-optical disk (for example, a compact disk, a digital versatile disk) Discs, Blu-ray® disks, smart cards, flash memory devices (eg cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppies (registered trademarks) ) Disks, magnetic strips, databases, servers and other suitable storage media. The program may also be transmitted from the network. The program may also be transmitted from the communication network via a telecommunication line.

(21) Each of the above-described aspects includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (registered trademark). , GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark) ), Other systems that utilize suitable systems and / or next-generation systems that are extended based on them.

(22) In each of the above aspects, the information, signals, etc. described may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.
In addition, the terms described in the present specification and / or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

(23) Each of the functions illustrated in FIGS. 1, 7, and 10 is realized by any combination of hardware and software. In addition, each function may be realized by a single device, or may be realized by two or more devices configured as separate bodies from each other.

(24) The programs exemplified in each of the above-described embodiments are called instructions, instruction sets, codes, code segments regardless of whether they are called software, firmware, middleware, microcode or hardware description language, or by other names. , Program code, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, execution threads, procedures or functions, etc. should be broadly interpreted to mean.
Further, software, instructions, and the like may be transmitted and received via a transmission medium. For example, the software uses wired technology such as coaxial cable, fiber optic cable, twist pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

(25) In each of the above-described embodiments, the information, parameters, etc. may be represented by absolute values, relative values from a predetermined value, or other corresponding information. May be good.

(26) The names used for the above-mentioned parameters are not limited in any respect. Further, mathematical formulas and the like using these parameters may differ from those expressly disclosed herein.

(27) In each of the above-described embodiments, the user device 1 includes a case where it is a mobile station. Mobile stations can be subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, mobile terminals, wireless, depending on the trader. It may also be referred to as a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

(28) In each of the above embodiments, the phrase "based on" does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

(29) Any reference to elements using designations such as "first", "second" as used herein does not generally limit the quantity or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must somehow precede the second element.

(30) As long as "inclusion," "comprising," and variations thereof in each of the embodiments described above are used herein or within the scope of the claims, these terms are used. As with the term "prepare", it is intended to be comprehensive. Furthermore, the term "or" as used herein or in the claims is intended not to be an exclusive OR.

(31) In the whole of the present application, if articles are added by translation, for example, a, an and the in English, unless the context clearly indicates that these articles are not. Including multiple.

(32) It will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modifications and modifications without departing from the gist and scope of the present invention, which is determined based on the description of the scope of claims. Therefore, the description herein is for illustrative purposes and has no limiting implications for the present invention. In addition, a plurality of aspects selected from the aspects illustrated in the present specification may be combined.

1,1a, 1c ... user device, 8 ... sound collector, 21, 21C ... acquisition unit, 22 ... transmission unit, 25, 25a ... estimation unit, 26 ... output unit, 251 ... voice recognition processing unit, 252 ... first Estimating unit, 253 ... 2nd estimation unit, 254, 254a ... Emotion estimation unit, ED ... Emotion data, GE1 ... Positive group, GE2 ... Negative group, GE3 ... Excitement group, GE4 ... Non-excitement group, SD ... Recognition character string, VD ... Voice data.

Claims (8)

An acquisition unit that acquires voice data indicating sounds including the speaker's voice, and
Based on the recognition character string obtained by subjecting the voice data to voice recognition processing, the speaker's emotions are either the first group to which the positive emotions belong or the second group to which the negative emotions belong. The first estimation part that estimates whether it belongs to
Based on the sound features indicated by the voice data, it is estimated whether the speaker's emotion belongs to the third group to which the emotion during excitement belongs or the fourth group to which the emotion not during excitement belongs. The second estimation part and
An emotion estimation unit that estimates the emotion of the speaker based on the estimation result of the first estimation unit and the estimation result of the second estimation unit.
Emotion estimator equipped with. The emotion estimation unit
When the estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the first group, it is estimated that the emotion of the speaker is joy.
The estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the second group, and the estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the third group. If shown, it is presumed that the speaker's emotions are anger,
The estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the second group, and the estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the fourth group. When indicated, the emotion of the speaker is presumed to be sadness,
The emotion estimation device according to claim 1. The emotion estimation unit
When the estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the fourth group, it is estimated that the emotion of the speaker is sadness.
The estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the third group, and the estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the first group. When showing, it is presumed that the speaker's emotions are joy, and
The estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the third group, and the estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the second group. When indicated, it is presumed that the speaker's emotions are anger.
The emotion estimation device according to claim 1. The first estimation unit
When the estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the third group, the emotion classification data for classifying the character string into any of joy, anger, and sadness is referred to. By comparing the character string of joy or anger included in the emotion classification data with the recognition character string, it is estimated whether the emotion of the speaker belongs to the first group or the second group. ,
The emotion estimation device according to claim 3. The emotion estimation unit
The estimation result of the first estimation unit indicates that the emotion of the speaker belongs to the first group, and the estimation result of the second estimation unit indicates that the emotion of the speaker belongs to the fourth group. If indicated, determine that it cannot be estimated,
The emotion estimation device according to claim 1. It is provided with an output unit that performs a process of adding a figure that embodies the emotion estimated by the emotion estimation unit to the recognition character string.
When the emotion estimation unit determines that the emotion estimation unit cannot estimate, the output unit includes a graphic character string obtained by adding a graphic that embodies joy to the recognition character string, and the recognition character. Outputs at least one graphic character string out of the graphic character string obtained by adding a graphic that embodies sadness to the column.
The emotion estimation device according to claim 5. An emotion estimation system including the emotion estimation device according to any one of claims 1 to 5 and a terminal device capable of communicating with the emotion estimation device.
The terminal device is
A sound collecting unit that collects sounds including the voice of the speaker, and
A transmission unit that transmits the voice data indicating a sound including the voice of the speaker to the emotion estimation device, and
A receiving unit that receives the recognition character string and emotion data indicating the emotion of the speaker estimated by the emotion estimation unit from the emotion estimation device.
An output unit that outputs data obtained by processing the recognition character string according to the emotion indicated by the emotion data, and
Emotion estimation system with. Acquires voice data indicating the sound including the speaker's voice,
Based on the recognition character string obtained by subjecting the voice data to voice recognition processing, the speaker's emotions are either the first group to which the positive emotions belong or the second group to which the negative emotions belong. Estimate whether it belongs to
Based on the sound features indicated by the voice data, it is estimated whether the speaker's emotion belongs to the third group to which the emotion during excitement belongs or the fourth group to which the emotion not during excitement belongs. ,
An estimation result indicating whether the speaker's emotion belongs to the first group or the second group, and whether the speaker's emotion belongs to the third group or the fourth group are shown. The emotion of the speaker is estimated based on the estimation result.
An emotion estimation method in which a computer performs processing.

Images