JP6720520B2 - Emotion estimator generation method, emotion estimator generation device, emotion estimation method, emotion estimation device, and program - Google Patents

Description

The present invention relates to an emotion estimator generation method, an emotion estimator generation device, an emotion estimation method, an emotion estimation device, and a program.

A technique for estimating the emotion of a speaker is being developed by using an emotion estimation device generated by machine learning using a speech data group obtained by labeling emotions as teacher data. For example, Patent Document 1 discloses a technique of obtaining a change amount of each of a voice intensity, a voice tempo, and a voice intonation, and estimating a speaker's emotion based on the obtained change amount.

JP, 2002-91482, A

Generally speaking, speaking in a state of excitement tends to be faster and louder than in normal speech. Further, when the user speaks in a disappointed state, his/her speaking style tends to be slower and the voice tends to be lower than that in normal speech. In this way, the emotion of the speaker at the time of utterance and the feature amount of the voice are correlated. Patent Document 1 discloses a technique for estimating the emotion of a speaker by analyzing such a change in the feature amount of voice data.

By the way, when comparing the feature amount of the voice uttered in the normal emotional state and the feature amount of the voice uttered in the angry emotional state, the tendency of the change in the feature amount may be different between the short words and the long words. is there. For example, short words that are easy to speak often have large changes in the feature amount of the voice depending on the emotional state at the time of speech. On the other hand, for a long word that is difficult to speak, such as a quick spoken word, the change in the voice feature amount may be small depending on the emotional state at the time of speech. Since the technique disclosed in Patent Document 1 uniformly estimates a speaker's emotion by words that have little change in speech feature amount and words that have large change depending on the emotional state at the time of utterance, the estimation accuracy increases. There was a problem that it was difficult.

The present invention has been made in view of such circumstances, and an emotion estimator generation method, an emotion estimator generation device, and an emotion estimation method that can improve the estimation accuracy of estimating the emotion of a speaker from voice data. , An emotion estimation device and a program are provided.

In order to achieve the above object, the emotion estimator generation method according to the first aspect of the present invention is
An analysis interval setting step of setting an analysis interval for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion A change pattern determination step of determining a change pattern of the feature amount,
Audio data classified for each change pattern of the feature quantity corresponding to each of the emotion as teacher data, and emotion estimator generation step of generating an emotion estimator for estimating a speaker emotion when the uttered voice ,
It is characterized by including.

Further, the emotion estimator generation device according to the second aspect of the present invention is
Analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining as a change pattern of the feature amount,
Audio data classified for each change pattern of the feature quantity corresponding to each of the emotion as teacher data, and emotion estimator generating means for generating an emotion estimator for estimating a speaker emotion when the uttered voice ,
It is characterized by including.

A program according to the third aspect of the present invention is
An analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data by a computer,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining the change pattern of the feature amount,
The feature quantity of the speech data classified for each change pattern as teacher data, the emotion estimator generating means for generating an emotion estimator for estimating a speaker emotion when the uttered voice corresponding to each of the emotion,
It is characterized by making it function as.

The emotion estimation method according to the fourth aspect of the present invention is
An analysis section setting step for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states to be analyzed,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion A change pattern determination step of determining a change pattern of the feature amount,
For each change pattern of the feature amount corresponding to each emotion, the emotion of the speaker at the time of uttering a voice, which is generated based on the teacher data having the change pattern of the same feature amount as the change pattern of the feature amount , Using an emotion estimator to estimate, an emotion estimation step of estimating the emotion of the speaker when the voice of the analysis section is uttered,
It is characterized by including.

Further, the emotion estimation device according to the fifth aspect of the present invention is
Analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states to be analyzed,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining as a change pattern of the feature amount,
For each change pattern of the feature amount corresponding to each emotion, the emotion of the speaker at the time of uttering a voice, which is generated based on the teacher data having the change pattern of the same feature amount as the change pattern of the feature amount , Using an emotion estimator for estimating, an emotion estimating means for estimating the emotion of the speaker when the voice of the analysis section is uttered,
It is characterized by including.

According to the present invention, it is possible to improve the estimation accuracy for estimating the emotion of the speaker from the voice data.

It is a block diagram which shows the physical constitution of the emotion estimator production|generation apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the functional structure of the emotion estimator production|generation apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating a morpheme. It is a figure for demonstrating a mora area. It is a figure for demonstrating the analysis method of a feature-value. It is a figure for demonstrating the analysis method of a feature-value. It is a figure for explaining classification. It is a figure for demonstrating the image of the identification threshold value of the produced emotion estimation apparatus. It is a block diagram which shows the functional structure of an emotion estimation apparatus. It is a flow chart for explaining generation processing of an emotion estimator. It is a flowchart for explaining an emotion estimation process. FIG. 8 is a diagram for explaining an analysis section of a feature amount according to Modification Example 1. FIG. 10 is a diagram for explaining a method of analyzing a feature amount according to Modification Example 1. FIG. 11 is a diagram for explaining analysis of a feature amount based on the strength of a voice according to Modification 2. FIG. 16 is a diagram for explaining a technique of estimating the degrees of a plurality of emotions according to Modification 5;

Hereinafter, an emotion estimator generation method, an emotion estimator generation device, an emotion estimation method, an emotion estimation device, and a program according to an embodiment of the present invention will be described with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals.

(Embodiment 1)
In the present embodiment, an emotion estimator generation device that generates an emotion estimator that estimates the emotion of the speaker from the voice data will be described, and then an emotion estimation device that estimates the emotion of the speaker when the voice is uttered will be described. .. In the present embodiment, the emotion estimation device is in a state of being sad (sorrowful), bored (bored), angry (angry), surprised (surprised), or discouraged by the speaker's emotions. A case will be described in which it is estimated that the emotional state is one of the seven basic emotional states: a state of being depressed (discouraged), a state of having disgust (dislike), and a state of being pleased (joy).
It should be noted that in the following embodiments, a change pattern of the feature amount of the audio data is called an accent type.

The emotion estimator generation device 100 according to the first embodiment physically includes a control unit 1, a storage unit 2, an input/output unit 3, and a bus 4, as illustrated in FIG. 1.

The control unit 1 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and a CPU (Central Processing Unit). The ROM stores an emotion estimator generation program according to this embodiment, an initial program for performing various initial settings, hardware inspection, program loading, and the like. The RAM functions as a work area for temporarily storing various software programs executed by the CPU, data necessary for executing these software programs, and the like. The CPU is a central processing unit that executes various processes and operations by executing various software programs.

The storage unit 2 includes a non-volatile memory such as a hard disk drive and a flash memory. The storage unit 2 stores voice data or the like used as teacher data.

The input/output unit 3 includes a voice input device for acquiring voice data to be teacher data, a CD (Compact Disc) drive, and a USB (Universal Serial Bus) interface. The input/output unit 3 acquires voice data to be teacher data. The input/output unit 3 also outputs a program for the generated emotion estimator or a parameter that determines the characteristics of the emotion estimator to an external device.

The bus 4 connects the control unit 1, the storage unit 2, and the input/output unit 3.

Functionally, the emotion estimator generation device 100 has a voice data acquisition unit 110, an analysis section setting unit 120, an accent type determination unit 130, a feature amount extraction unit 140, and an emotion estimation as shown in FIG. And a container generator 150. The analysis section setting unit 120 also includes a morpheme analysis unit 121 and an accent phrase extraction unit 122. Further, the accent type determination unit 130 includes a mora section extraction unit 131 and an accent type extraction unit 132.

The voice data acquisition unit 110 acquires voice data used as teacher data for generating an emotion estimator via the input/output unit 3. The teacher data is composed of, for example, voices uttered in seven emotional states: sadness, boredom, anger, surprise, discouragement, disgust, and joy. In addition, the teacher data is composed of voice data containing a sufficiently large number of types of words and phrases. It is preferable that the number of utterers of the teacher data and the types of accent phrases included in the teacher data are large. An accent phrase is a unit that divides voice data that combines a noun and a particle, or a verb and an auxiliary verb. For example, as the teacher data, voice data including 1000 or more kinds of accent phrases that a large number of about 500 people uttered in 7 kinds of emotional states is prepared.

The analysis section setting unit 120 sets an analysis section, which is a unit for analyzing the characteristics of the voice data as the teacher data. Therefore, the analysis section setting unit 120 includes a morpheme analysis unit 121 and an accent phrase extraction unit 122.

The morpheme analysis unit 121 divides the acquired voice data into morphemes. A morpheme is a minimum unit having a meaning as a language. For example, as shown in FIG. 3, the voice "Bow draws a picture of a bōbo in a folding screen" is shown in FIG. 3, "Bow", "ga", "screen", "ni", "good", "ni". , "Shaved", "no", "picture", "wo", "draw", "ta" are divided into 12 morphemes.

The accent phrase extraction unit 122 extracts an accent phrase from the acquired voice data. An accent phrase is a section in which a noun or verb divided into morphemes is followed by a particle or auxiliary verb. In the above-mentioned example, the accent phrases are "shaved", "in the folding screen", "well", "shaved", "picture", and "painted". In the present embodiment, a case will be described in which the characteristics of voice data are analyzed in units of accent phrases. The reason why the voice data is analyzed in units of accent phrases is that the emotional state of the speaker often changes in units of accent phrases.

The accent type determination unit 130 determines the accent type of the accent phrase. The accent type is "H" when the feature amount of the speech is larger than the average feature amount and "L" when the feature amount of the speech is smaller than the average feature amount for each mora section in which the syllables forming the accent phrase are spoken. Is a pattern of a combination of "H" and "L". To determine the accent type, the accent type determination unit 130 includes a mora section extraction unit 131 and an accent type extraction unit 132.

As shown in FIG. 4, the mora section extraction unit 131 extracts a mora section from the voice data of the accent phrase section to be analyzed. The mora section is a section in which one syllable is uttered. Explaining in the case of the accent phrase "Bozu ga", each syllable of "BO", "U", "Z", and "GA" is a mora section.

The accent type extraction unit 132 determines whether each of the mora sections corresponds to “H” or “L”, and extracts the accent type of the accent phrase. As an accent type extraction method, there is a method of using a feature amount such as voice intensity, voice pitch, and phoneme utterance time length. Here, an extraction method focusing on the pitch of voice will be described with reference to FIGS. 5 and 6.

As shown in FIG. 5, the accent type extraction unit 132 sets a time window of a predetermined time that further subdivides the mora section. The window 1 is set for one mora section, and the voice data in the window is subjected to FFT (Fast Fourier Transform) conversion. Next, the audio data in the window 2 which is the window 1 shifted by a predetermined time dt is subjected to FFT conversion. Thereafter, the audio data in the window n is similarly FFT-converted. As a method of setting the time window, for example, the time width dt shifted from the time window is set so that 10 or more time windows are formed in the mora section. This is because if the number of time windows is too small, the calculation accuracy will decrease.

FIG. 6 is an example showing a spectral distribution of audio data in each window obtained by the above FFT conversion. The horizontal axis represents frequency and the vertical axis represents spectrum intensity. The peak frequency existing in the lowest frequency region in this spectrum is f0. This f0 represents a speaker-specific fundamental frequency obtained from the voice data in the window section. Let f0 obtained from the window 1 be f0_1 and f0 obtained from the window 2 be f0_2. Similarly, f0 obtained from the window n is set to f0_n. Then, the accent type extraction unit 132 calculates the average value from f0_1 to f0_n and sets it as the average fundamental frequency 1_f0 of the first mora section.

The accent type extraction unit 132 performs the same calculation for all mora sections included in the accent phrase. The average fundamental frequency m_f0 in the m-th mora section can be calculated using Expression 1.

m_f0=1/n・Σf0_n (Equation 1)

Next, the accent type extraction unit 132 obtains the average fundamental frequency m_th in the accent phrase section using Expression 2.

m_th={max(1_f0,...,n_f0)−min(1_f0,...,n_f0)}/2 (Equation 2)

Next, the accent type extraction unit 132 compares the average fundamental frequency m_f0 of the mora section with the average fundamental frequency m_th of the accent phrase section, and if m_f0≧m_th, “H”, and if m_f0<m_th, “L”. Is assigned to each mora section. The accent type extraction unit 132 extracts the accent type formed by the combination of H and L by adding “H” and “L” to each mora section that forms the accent phrase in this way.

The accent type determination unit 130 performs this process for all the accent phrases generated from the teacher data. The example of the accent type shown in FIG. 7 is an example in which the number of mora sections is 6 or less, in which 20 types of accent types are selected in descending order of occurrence frequency among accent types obtained from a large amount of teacher data. When the number of mora sections included in the accent phrase to be analyzed is 6 or less, class names from class 1 to class 20 are sequentially assigned to these 20 types of accent types. There is a one-to-one correspondence between accent types and classes. The accent type determination unit 130 associates the accent phrase unit (voice data) as the teacher data with the accent type (class) and stores them in the storage unit 2. Note that accent phrases that do not correspond to these 20 types of accent types are excluded from the teacher data. Hereinafter, in this embodiment, a case where the number of mora sections included in the accent phrase to be analyzed is 6 or less will be described.

Here, 20 kinds of accent type selection methods when the number of mora sections included in the accent phrase is 6 or less are generated based on an experimental result obtained by statistically processing a large amount of Japanese uttered with 7 kinds of emotions. 20 types of accent types are selected in descending order of frequency. If the number of accent type classes is reduced, the frequency of teacher data that does not correspond to the accent type increases, and the estimation accuracy of the emotion estimation device having the generated emotion estimator decreases. On the other hand, if the number of accent-type classes is increased, the number of types of emotion estimators to be generated increases, which increases the manufacturing cost of the emotion estimation device. Therefore, the number of accent type classes is determined based on these two tradeoffs. Note that the number of accent types is 20 in the case of Japanese, where the number of mora sections included in the accent phrase is 6 or less. When the number of mora sections included in the accent phrase is 7 or more, or in the case of another language, it is necessary to further determine the frequency of accent type occurrence by statistical processing.

Returning to FIG. 2, the feature amount extraction unit 140 extracts the voice feature amount for each accent phrase. The feature amount of the voice is the volume of the voice, the pitch of the voice, the utterance time length of the phoneme, and the like. Then, the extracted feature quantities are given class names from class 1 to class 20 determined by the accent type determination unit 130, and stored in the storage unit 2 as teacher data.

The emotion estimator generation unit 150 acquires teacher data for each class from the storage unit 2 and generates an emotion estimator adapted to each class. Specifically, the emotion estimator generation unit 150 acquires the teacher data classified into class 1 and sets the emotional state at the time of utterance of the teacher data in seven sad emotional states (sadness). , Bored (bored), angry (angry), surprised (surprised), discouraged (disappointed), disgusted (disliked), pleased An emotion estimator for class 1 classified into (joy) is generated. FIG. 8 is an image diagram in which a discrimination threshold for discriminating between seven types of emotions is two-dimensionally expressed. A known technique can be used for the method of generating the emotion estimator based on the teacher data. Next, the emotion estimator generation unit 150 acquires the teacher data classified into class 2, and determines the emotional state at the time of utterance of the teacher data as seven types of emotional states: sadness, boredom, anger, surprise, and discouragement. Generates an emotion estimator for class 2, which is classified into hate, hate, and happiness. Similarly, the emotion estimator generation unit 150 generates emotion estimators up to class 20.

Next, the configuration of the emotion estimation device 200 incorporating the 20 types of emotion estimators generated by the above description will be described with reference to FIG. The emotion estimation device 200 is physically the same as the configuration shown in FIG.

The ROM included in the control unit 1 stores the emotion estimation program generated by the emotion estimator generation device 100 according to this embodiment. The storage unit 2 stores voice data to be analyzed. The input/output unit 3 includes a voice input device for obtaining voice data to be analyzed, a CD drive, and a USB interface. Further, the input/output unit 3 may acquire the parameter that determines the characteristic of the emotion estimator generated by the emotion estimator generation device 100. The input/output unit 3 also includes a display device or a voice output device for outputting the result of emotion estimation.

As shown in FIG. 9, the emotion estimation device 200 includes a voice data acquisition unit 210, a speaker division unit 220, an analysis section setting unit 230, an accent type determination unit 240, a selection unit 250, and a feature amount extraction unit. The functions of 260, the emotion estimation unit 270, and the integration unit 280 are included. The analysis section setting unit 230 also includes the functions of the morpheme analysis unit 231 and the accent phrase extraction unit 232. Further, the accent type determination unit 240 includes the functions of the mora section extraction unit 241 and the accent type extraction unit 242.

The voice data acquisition unit 210 acquires the voice that is the analysis target and is uttered by the user. The voice data acquisition unit 210 is composed of a voice acquisition device such as a microphone. Further, the audio data acquisition unit 210 includes a CD drive and a USB interface, and can also acquire a user's audio as audio data.

The speaker division unit 220 divides the acquired voice data to be analyzed for each speaker. This is because when the voice data of a plurality of persons is present in the voice data, the emotion of the speaker is estimated for each sentence uttered by one speaker. A method of dividing the voice data into speakers is performed by using a known technique. For example, the division can be performed based on the correlation between the voice intensity, the voice pitch, the phoneme utterance time length, and the like.

The analysis section setting unit 230 and the accent type determination unit 240 have the same configuration as the emotion estimator generation device 100 in order to analyze the voice data to be analyzed under the same conditions as when the emotion estimator was generated. That is, the analysis section setting unit 230 extracts the accent phrase from the voice data, and the accent type determination unit 240 determines the accent type (class) for each accent phrase.

The selection unit 250 selects an emotion estimator corresponding to the corresponding class for each accent phrase classified into classes. Specifically, the emotion estimator corresponding to the accent phrase class to be analyzed is selected from the 20 types of emotion estimators built into the emotion estimation device 200.

The feature amount extraction unit 260 has the same configuration as the emotion estimator generation device 100, and extracts the feature amount from the voice data under the same conditions. Then, the feature amount extraction unit 260 stores the extracted feature amount and the class name indicating the accent type in the storage unit 2 in association with each other.

The emotion estimation unit 270 estimates the emotion of the speaker for each accent phrase using the emotion estimator selected by the selection unit 250. Specifically, when estimating the emotion of the accent phrase classified into class 1, the emotion estimation unit 270 selects the emotion estimator for class 1 and estimates the emotion of the speaker. When estimating the emotion of the accent phrase classified into class n, emotion estimating section 270 selects the emotion estimator for class n and estimates the emotion of the speaker. Then, the emotion estimation unit 270 estimates whether the emotional state of the speaker when speaking the accent phrase to be analyzed corresponds to the emotional state of sadness, boredom, anger, surprise, discouragement, disgust, or joy. To do.

The integration unit 280 estimates the emotion of the speaker for each sentence of voice data. Specifically, the integration unit 280 estimates the most common emotion in one sentence as the emotion of the speaker who uttered the sentence. For example, "Bozu is a folding screen," which consists of seven accent phrases: "Bozu ga", "Byobu ni", "I am good", "Bozu no", "Picture", "Draw", and "Ta". In the sentence "I drew a picture of a shaved", the number of accent phrases judged as "joy" was 4, the number of accent phrases judged as "anger" was 2, and it was judged as "surprise". When the number of accent phrases is 1, the most "joy" is estimated as the emotion of the speaker when the "shaved the picture of the shaved head by the shaved head" is uttered.

Next, a process in which the emotion estimator generation device 100 having the above configuration generates an emotion estimator will be described with reference to FIG. 10. It is assumed that the voice data used as teacher data, which is uttered in seven emotional states of sadness, boredom, anger, surprise, discouragement, disgust, and joy, is stored in advance in the storage unit 2. It is assumed that the number of mora sections of the accent phrase included in the voice data to be analyzed is 6 or less. The person in charge of generating the emotion estimator starts the emotion estimator generation program installed in advance in the emotion estimator generation device 100 to start the flowchart shown in FIG. 10.

When the emotion estimator generation program is activated, the control unit 1 acquires the teacher data stored in the storage unit 2 into the voice data acquisition unit 110 (step S11). Then, the morpheme analysis unit 121 divides the acquired voice data in units of morphemes (step S12). Next, the accent phrase extraction unit 122 extracts an accent phrase, which is a unit for analyzing the characteristics of the voice data, and divides the voice data into accent phrases (step S13).

Next, the mora section extraction unit 131 extracts the mora section included in the accent phrase (step S14). Then, the accent type extraction unit 132 extracts the accent type of the accent phrase (step S15). Specifically, the accent type extraction unit 132 classifies the accent phrases used as teacher data into 20 classes, as described with reference to FIG. 7 (step S16). In order to make the classification, the accent type extraction unit 132 compares the average fundamental frequency m_f0 of each mora section with the average fundamental frequency m_th of the accent phrase section, as described with reference to FIGS. 5 and 6, and m_f0 If ≧m_th, “H” is given to each mora section, and if m_f0<m_th, “L” is given to each mora section. The accent type extraction unit 132 assigns H and L to the accent phrase used as the teacher data in this way for each mora section forming the accent phrase, and extracts the accent type from the H and L pattern. Then, the accent type determination unit 130 determines the accent type of the teacher data to be one of the 20 accent types (classes) shown in FIG. 7.

Next, the feature amount extraction unit 140 extracts the feature amount of the voice data to be the teacher data for each accent phrase, and associates the extracted feature amount data with the classified class in the storage unit 2 as the teacher data. It is stored (step S17).

The emotion estimator generation unit 150 generates an emotion estimator for each class based on the teacher data classified by accent type (class) (step S18). Specifically, the emotion estimator generation unit 150 acquires the teacher data (accent phrase) classified into the class 1, and sets the emotional state at the time of utterance of the teacher data as seven types of emotional states, sadness, Generates an emotion estimator that can be categorized as bored, angry, surprised, discouraged, hateful, and joyful. More specifically, an identification threshold value (parameter of a mathematical expression forming a classifier) for classifying into seven types of emotions as shown in FIG. 8 is generated. Next, the emotion estimator generation unit 150 acquires the teacher data (accent phrase) classified into the class 2 and sets the emotional state at the time of utterance of the teacher data as seven emotional states of sadness, boredom, Generate a second emotion estimator that can be classified as anger, surprise, discouragement, disgust, or joy. The emotion estimator generation unit 150 thus generates 20 types of emotion estimators. This is the end of the description of the emotion estimator generation processing of the emotion estimator generation device 100.

Next, an emotion estimation process in which the emotion estimation device 200 having 20 types of emotion estimators generated by the emotion estimator generation device 100 estimates the emotion of the speaker will be described with reference to FIG. 11. The user starts the emotion estimation program preinstalled in the emotion estimation device 200 and inputs the voice data to be analyzed into the emotion estimation device 200, whereby the flowchart shown in FIG. 11 is started.

The control unit 1 starts the emotion estimation program, and when the user inputs the voice data to be analyzed into the emotion estimation apparatus 200, the speaker division unit 220 divides the acquired voice data into speakers, and the storage unit 2 Remember. Next, the analysis section setting unit 230 acquires voice data from the storage unit 2 for each speaker (step S31). Next, the morpheme analysis unit 231 decomposes the voice data of an arbitrary speaker into morphemes (step S32), and the accent phrase extraction unit 232 determines an accent phrase as an analysis unit (step S33).

Next, the accent type determination unit 240 determines the accent type (class) for each accent phrase, as in the operation description of the emotion estimator generation device 100. Specifically, the mora section extraction unit 241 extracts the mora section included in the corresponding accent phrase (step S34), and the accent type extraction unit 242 extracts the accent type of the accent phrase (step S35). Then, the accent type determination unit 240 determines the class to which the accent phrase belongs from the extracted accent type. Then, the selection unit 250 is an emotion estimator generation device 100 based on the same accent type (class) teacher data as an emotion estimator used to estimate the emotion of the speaker when the corresponding accent phrase is uttered. The emotion estimator generated by is selected (step S36).

On the other hand, the feature amount extraction unit 260 extracts voice feature amounts such as the voice intensity, voice pitch, and phoneme duration of the accent phrase to be analyzed, and associates the extracted feature amount with the determined class. It is stored in the storage unit 2 (step S37).

Next, the emotion estimation unit 270 estimates the emotion of the speaker when the relevant accent phrase is uttered, using the emotion estimator selected by the selection unit 250 (step S38).

When emotion estimation is completed for one of the accent phrases, the emotion estimation device 200 determines whether or not there is an accent phrase whose analysis has not been completed (step S39). If there is an accent phrase that has not been analyzed (step S39: No), another accent phrase that has not been analyzed is extracted (step S40), and the emotion corresponding to that accent phrase is estimated.

When the analysis of all accent phrases is completed (step S39: Yes), the emotion estimation device 200 performs integration processing for each analyzed sentence (step S41). Specifically, the integration unit 280 estimates the most common emotion as the emotion of the speaker when the sentence is uttered, based on the emotion estimation result for each accent phrase included in the sentence to be analyzed.

Next, the emotion estimation device 200 determines whether or not the analysis has been completed for all the sentences that the initially acquired arbitrary person uttered (step S42). When the analysis is not completed for all the sentences (step S42: No), another sentence is extracted (step S43), and the emotion estimation process is continued for the other sentences.

On the other hand, if the emotion estimation apparatus 200 has completed the analysis for all the sentences (step S42: Yes), the emotion estimation apparatus 200 determines whether or not the emotion estimation has been completed for all the persons included in the voice data. (Step S44). If the analysis has not been completed for all persons (step S44: No), the voice data of other persons are extracted and the emotion estimation process is continued (step S45). If the analysis process has been completed for all persons (step S44: Yes), the emotion estimation process ends.

As described above, the emotion estimator generation device 100 generates the emotion estimator for each accent type based on the teacher data classified for each accent type. Then, the emotion estimation device 200 estimates the emotion of the speaker using the emotion estimator generated for each accent type. Specifically, the emotion estimation device 200 classifies the voice data to be analyzed for each accent type, and estimates the emotion of the speaker using the emotion estimator generated based on the teacher data having the same accent type. .. This can improve the estimation accuracy for estimating the emotion of the speaker from the voice data.

Further, since the accent type extraction unit 132 extracts the change in the feature amount of the voice in the unit of the mora section, the emotion estimator generation device 100 uses the emotion estimator that can analyze the emotion of the speaker more finely. Can be generated.

The accent type extraction unit 132 also extracts an accent type based on a change in the fundamental frequency of the voice. The fundamental frequency of voice tends to change depending on the emotional state at the time of utterance. Therefore, the emotion estimator generation device 100 can generate an emotion estimator that can more accurately estimate the emotion of the speaker. Further, for the same reason, the emotion estimation device 200 can estimate the emotion of the speaker more accurately.

Since the analysis interval setting unit 120 analyzes the voice in units of morphemes, the emotion estimator generation device 100 can generate an emotion estimator that can analyze the emotion of the speaker more accurately.

The emotion estimator generation device 100 generates an emotion estimator that classifies emotions of a speaker during utterance into sadness, boredom, anger, surprise, discouragement, disgust, and joy. The emotion estimation device 200 incorporating this estimator can classify the emotional state of the speaker at the time of utterance into sadness, boredom, anger, surprise, discouragement, disgust, and joy.

(Modification 1)
In the first embodiment, the feature amount is analyzed in units of mora sections in order to determine the accent type. In the first modified example, the feature amount is analyzed only in the vowel section in the mora section. Specifically, as shown in FIG. 12, the voice data of only the vowel section is extracted and the characteristic amount is analyzed as shown in FIG. The method of analyzing the fundamental frequency is the same as that described in the first embodiment.

The reason for focusing only on the vowel section is that the vowel section has a longer phoneme duration than the consonant section, and the energy of the included speech is also larger. This is because it appears significantly in the section.

As described above, the emotion estimation device 200 according to the first modification can improve the estimation accuracy of emotion estimation by analyzing the feature amount only in the vowel section.

(Modification 2)
In the description of the first embodiment, the case where the accent type extraction unit 132 uses the pitch information of the voice (fundamental frequency of the voice) as the feature amount of the voice has been described. In the second modification, a case will be described where voice intensity information is used as a voice feature amount. Here, a technique focusing on the fact that the energy distribution of the voice in the utterance section of the vowel changes depending on the emotional state at the time of utterance will be described.

Specifically, the accent type extraction unit 132 determines whether the energy peak of the voice indicated by a dotted circle in FIG. 14 exists in the first half or the second half of the vowel section. For example, "H" is given when the peak exists in the first half, and "L" is given when the peak exists in the latter half. Thereby, the accent type determination unit 130 determines the accent type. When categorizing accent types according to the strength of voice, it is necessary to examine the classification method based on experimental data. The other description is the same as that of the first embodiment.

In the description of the second modification, the analysis method focusing on the change in the time position of the peak point of the voice energy has been described, but the accent type can also be extracted by using the change in the voice intensity. This tendency is used because the speech intensity tends to be high when speaking in an angry state and tends to be low when speaking in a sad state. In this case, for example, the peak intensity of the voice for each mora section included in the accent phrase section is measured to obtain the average peak strength of the accent phrase section. Then, the accent type can be extracted by comparing the peak intensity and the average peak intensity of the sound in the mora section and adding "H" or "L" to each mora section.

As described above, the emotion estimation device 200 according to the second modification analyzes the emotional state of the voice when the voice is uttered, using the change information of the intensity of the voice, so that the estimation accuracy of the emotion estimation can be improved.

(Modification 3)
In the third modification, a case will be described where the phoneme duration is used as the feature amount of the voice. If you speak in an angry or pleased state, phoneme duration tends to be short, and if you speak in a boring or sad state, phoneme duration tends to be long. is there.

Specifically, the accent type extraction unit 132 compares the duration of the vowel included in the mora section with the average duration of the same vowel included in the teacher data, and the duration of the vowel included in the mora section. If the length is longer than the average duration time, “H” is given, and if the length is shorter, “L” is given, whereby the accent type determination unit 130 determines the accent type.

In the description of the first embodiment, the first modified example, and the second modified example, the average value of the feature amount of the voice in the accent phrase section that is the analysis section and the average value of the mora section are compared. However, when comparing phoneme durations, the accent type extraction unit 132 of the emotion estimator generation apparatus 100 determines that the average duration is not the average of the speech data in the analysis section, but the average duration of the entire teacher data. Compare with. Since vowels have different durations, they cannot be compared with different vowel durations. Since the number of the same vowels included in the accent phrase is small, the variation in the average duration time becomes large, which causes an erroneous determination. Therefore, it is preferable to average the entire teacher data.

On the other hand, it is preferable that the accent type extraction unit 242 of the emotion estimation device 200 calculates the average duration time for each vowel of the voice data for each speaker classified by the speaker classification unit 220.

When categorizing accent types by phoneme duration, it is necessary to examine the classification method based on experimental data. The other description is the same as that of the first embodiment.

As described above, the emotion estimation apparatus 200 according to the third modification analyzes the emotional state at the time of utterance of voice by using the change information of the utterance time length of the phoneme, so that the estimation accuracy of the emotion estimation can be improved. ..

(Modification 4)
In the first embodiment and the modified example 1, the technique of extracting the accent type by using the pitch information of the voice as the feature amount of the voice has been described. In addition, in the second modification, a technique for extracting the accent type by using the strength information of the voice is introduced, and in the third modification, a technique for extracting the accent type by using the phoneme duration is introduced. When extracting the accent type, these techniques can be used alone, but the accent type can also be extracted by combining two or more techniques such as voice pitch information and voice intensity information. The combination of two or more pieces of information increases the types of accent types, but the accuracy of emotion estimation can be improved.

In the above description, the strength of the voice, the pitch of the voice, and the duration of the phoneme are described as an example of the feature amount of the voice, but the feature amount is not limited thereto. For example, the accent type can be determined by extracting the amount of change in voice intensity, the amount of change in voice pitch, the amount of change in phoneme duration, and the like.

(Modification 5)
In the description of the first embodiment, based on the emotion estimation result for each accent phrase included in the analysis target sentence, a technique for estimating the most common emotion as the emotion of the speaker when the sentence is uttered has been described. .. However, the method of integration processing need not be limited to this. For example, it is possible to perform estimation including a plurality of emotions such as “joy with a little surprise”. In many cases, the classifier configuring the emotion estimator acquires the feature amount as a vector and determines which emotion to classify based on the distance between the vector and the identification threshold. For example, the feature quantities corresponding to seven accent phrases “Bou ga ga”, “Bibu ni ni”, “I am good at”, “Bozu no”, “Picture”, and “I drew” 1 to 7 shown in FIG. It is assumed that the average vector obtained by combining the seven position vectors is the position vector indicated by "average" in FIG. In this case, the position vector "average" belongs to the area of joy, but exists near the boundary between joy and surprise. In such a case, an emotion estimation result including a nuance that “a little surprised emotions may be mixed” may be output.

Since FIGS. 8 and 15 are two-dimensional image representations of a seven-dimensional identification space, it is difficult to represent a complicated example. However, it is possible to numerically calculate the distances from the respective discrimination boundaries in the classifier that constitutes the emotion estimator. Therefore, it is possible to numerically calculate a combination of a plurality of emotions such as “anger and sadness” and “anger and discouragement” and the degree of the emotion (distance from the discrimination boundary). Furthermore, by setting a plurality of thresholds, it is possible to estimate emotions including two or more emotions such as “anger, sadness, discouragement”. Moreover, the composite degree of a plurality of emotions can also be estimated.

By providing the configuration and processing described in the modification 5, the emotion estimator generation device 100 can generate an emotion estimator capable of estimating the degrees of a plurality of emotions. In addition, the emotion estimation device 200 having an emotion estimator capable of estimating a plurality of emotion levels can estimate a plurality of emotion levels of the speaker.

In the description of the first embodiment, the emotional state of the speaker is classified into seven types of sadness, boredom, anger, surprise, discouragement, disgust, and joy. However, the emotion classification method needs to be limited to this. There is no. For example, it may be classified into four types: happiness, anger, sadness, and comfort.

In addition, in the description of the first embodiment, a case where the emotional state of the speaker at the time of utterance is classified into any of seven emotional states of sadness, boredom, anger, surprise, discouragement, disgust, and joy will be described. I explained that teacher data that does not correspond to this is excluded. However, the emotional state of “normal” may be provided as the emotion of the speaker, and the teacher data that could not be classified into seven emotions may be classified into the emotion “normal”. Thereby, the emotion estimator generation device 100 can generate the emotion estimator capable of estimating the emotion of the speaker into eight emotions obtained by adding “ordinary” to the seven emotions. Further, the emotion estimation device 200 can estimate the emotion of the speaker into eight emotions obtained by adding “ordinary” to the seven emotions.

In the description of the first embodiment, the analysis section is described as the section unit of the accent phrase, but the analysis section need not be limited to this. For example, the analysis section may be a word utterance section, a breath paragraph section that is a breathing section, or a sentence utterance section. When the analysis section is the utterance section of a sentence, the integration unit 280 may estimate the emotion of the speaker in sentence units, or may further estimate the emotion of the speaker in a unit of a plurality of sentences. You may

Further, in the description of Expression 1, the process of representing the feature amount of the section by using the average value has been described, but the process may be performed by using the median value instead of the average value. Further, the lowest frequency may be used as the representative value for the processing.

Further, in the description of the equation 2, the processing in which the median value is used to represent the feature amount of the section has been described, but the processing may be performed using the average value instead of the median value.

Further, it is possible to provide the emotion estimator generation device 100 and the emotion estimation device 200 that are provided with the configuration for realizing the functions according to the present invention in advance, and by applying the program, existing personal computers, information terminal devices, etc. The emotion estimator generation device 100 and the emotion estimation device 200 according to the present invention can also be caused to function. That is, a program for realizing the functional configurations of the emotion estimator generation device 100 and the emotion estimation device 200 exemplified in the above-described embodiments can be executed by a CPU or the like that controls an existing personal computer, information terminal device, or the like. When applied, it can function as the emotion estimator generation device 100 and the emotion estimation device 200 according to the present invention. The emotion estimator generation method and the emotion estimation method according to the present invention can be implemented using the emotion estimator generation device 100 and the emotion estimation device 200.

Moreover, the application method of such a program is arbitrary. For example, the program can be stored in a computer-readable recording medium (CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical disc), etc.), and the like, and can also be applied to the Internet, etc. The program can also be applied by storing the program in a storage on the network and downloading the program.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the specific embodiments, and the present invention includes the inventions described in the claims and their equivalents. Be done. The inventions described in the initial claims of the present application will be additionally described below.

(Appendix 1)
An analysis interval setting step of setting an analysis interval for analyzing the feature amount of the voice data which is the source of the teacher data,
A change pattern determination for determining a pattern in which the feature amount of the voice data included in the analysis section changes, as a change pattern in the feature amount of the voice data included in the analysis section, based on the change patterns classified into a plurality of classes. Steps,
Emotion estimator generation that generates an emotion estimator that estimates the emotion of the speaker when the voice is uttered for each of the feature amount change patterns, using the voice data classified for each of the feature amount change patterns as teacher data. Steps,
Emotion estimator generation method including.

(Appendix 2)
The change pattern determination step,
A mora section extracting step of dividing the voice data included in the analysis section into mora sections that are syllable units;
Based on the comparison result of comparing the average value of the feature amount of the voice data in the analysis section and the average value of the feature amount of the voice data in the mora section for each mora section, the voice data of the analysis section is uttered. Change pattern extraction step of extracting a change pattern of the feature amount of the voice that changes for each mora section at the time,
The emotion estimator generation method according to appendix 1, further comprising:

(Appendix 3)
In the change pattern extracting step, the fundamental frequency of the voice extracted from the voice data is used as the feature amount of the voice, and the average fundamental frequency of the voice in the analysis section and the average fundamental frequency of the voice in the mora section are set for each mora section. In comparison with the above, when the average fundamental frequency of the sound in the mora section is higher than the average fundamental frequency of the sound in the analysis section, High is given, and when it is lower, Low is given, and the sound changes to High and Low for each mora section. The change pattern of the feature amount of
The emotion estimator generation method according to appendix 2, characterized in that.

(Appendix 4)
In the change pattern extraction step, the strength of the voice extracted from the voice data is used as the feature quantity of the voice, and the average strength of the voice in the analysis section and the average strength of the voice in the mora section are compared for each mora section. , If the average strength of the sound in the mora section is higher than the average strength of the sound in the analysis section, High is given, and if it is low, Low is given, and the change in the feature amount of the sound changing to High and Low for each mora section. Extract patterns,
The emotion estimator generation method according to appendix 2, characterized in that.

(Appendix 5)
In the change pattern extraction step, using the phoneme duration extracted from the voice data as a voice feature amount, the average duration of the phoneme in the analysis section, and the average duration of the phoneme in the mora section, Comparison is made for each mora section. When the average duration of the phonemes in the mora section is longer than the average duration of the phonemes in the analysis section, High is given, and when the average duration is short, Low is given. A change pattern of the feature amount of the voice that changes to Low is extracted,
The emotion estimator generation method according to appendix 2, characterized in that.

(Appendix 6)
In the change pattern extraction step, a change pattern of the voice feature amount is extracted by using at least one of the fundamental frequency of the voice, the voice intensity, and the duration of the phoneme as the voice feature amount.
The emotion estimator generation method according to any one of appendices 2 to 5, characterized in that

(Appendix 7)
In the analysis section setting step, the speech data is divided into morphemes, which are the smallest units having a language meaning, and a section of an accent phrase combined with a particle or auxiliary verb uttered after the morpheme is set as the analysis section. To do
7. The emotion estimator generation method according to any one of appendices 1 to 6, characterized in that.

(Appendix 8)
In the mora section extraction step, when the voice data is displayed in text, one kana character is set as one mora section, the small kana character is combined with the preceding kana character as one mora section, and the long sound is independent. 1 mora section,
The emotion estimator generation method according to appendix 2, characterized in that.

(Appendix 9)
The emotion estimator estimates the emotion of the speaker at the time of utterance to be any one of sadness, boredom, anger, surprise, discouragement, disgust, and joy.
The emotion estimator generation method according to any one of appendices 1 to 8, characterized in that.

(Appendix 10)
A change pattern setting step of setting change patterns classified into the plurality of classes,
10. The emotion estimator generation method described in any one of appendices 1 to 9, characterized in that.

(Appendix 11)
Analysis section setting means for setting an analysis section for analyzing the feature amount of the voice data which is the source of the teacher data,
A change pattern determination for determining a pattern in which the feature amount of the voice data included in the analysis section changes, as a change pattern in the feature amount of the voice data included in the analysis section, based on the change patterns classified into a plurality of classes. Means and
Emotion estimator generation that generates an emotion estimator that estimates the emotion of the speaker when the voice is uttered for each of the feature amount change patterns, using the voice data classified for each of the feature amount change patterns as teacher data. Means and
Emotion estimator generation device including.

(Appendix 12)
Analysis section setting means for setting an analysis section for analyzing a feature amount of voice data which is a source of teacher data by a computer;
A change pattern determination for determining a pattern in which the feature amount of the voice data included in the analysis section changes, as a change pattern in the feature amount of the voice data included in the analysis section, based on the change patterns classified into a plurality of classes. means,
Emotion estimator generation that generates an emotion estimator that estimates the emotion of the speaker when the voice is uttered for each of the feature amount change patterns, using the voice data classified for each of the feature amount change patterns as teacher data. means,
Program to function as.

(Appendix 13)
An analysis section setting step for setting an analysis section for analyzing the feature amount of the voice data to be analyzed,
A change pattern determination for determining a pattern in which the feature amount of the voice data included in the analysis section changes, as a change pattern in the feature amount of the voice data included in the analysis section, based on the change patterns classified into a plurality of classes. Steps,
Emotion for estimating the emotion of the speaker when the voice of the analysis section is uttered, using the emotion estimator generated for each change pattern of the feature amount based on the teacher data having the same change pattern of the feature amount An estimation step,
Emotion estimation method including.

(Appendix 14)
Analysis section setting means for setting an analysis section for analyzing the feature amount of the voice data to be analyzed,
A change pattern determination for determining a pattern in which the feature amount of the voice data included in the analysis section changes, as a change pattern in the feature amount of the voice data included in the analysis section, based on the change patterns classified into a plurality of classes. Means and
Emotion for estimating the emotion of the speaker when the voice of the analysis section is uttered, using the emotion estimator generated for each change pattern of the feature amount based on the teacher data having the same change pattern of the feature amount Estimation means,
Emotion estimation device equipped with.

1... Control unit, 2... Storage unit, 3... Input/output unit, 4... Bus, 100... Emotion estimator generation device, 110, 210... Voice data acquisition unit, 120, 230... Analysis section setting unit, 121, 231,... Morphological analysis unit, 122, 232... Accent phrase extraction unit, 130, 240... Accent type determination unit, 131, 241, Mora section extraction unit, 132, 242... Accent type extraction unit, 140, 260... Feature amount extraction unit, 150 ... emotion estimation device generation unit, 200... emotion estimation device, 220... speaker division unit, 250... selection unit, 270... emotion estimation unit, 280... integration unit

Claims (14)

An analysis interval setting step of setting an analysis interval for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion A change pattern determination step of determining a change pattern of the feature amount,
Audio data classified for each change pattern of the feature quantity corresponding to each of the emotion as teacher data, and emotion estimator generation step of generating an emotion estimator for estimating a speaker emotion when the uttered voice ,
Emotion estimator generation method including. The change pattern determination step,
A mora section extracting step of dividing the voice data included in the analysis section into mora sections that are syllable units;
Based on the comparison result of comparing the average value of the feature amount of the voice data in the analysis section and the average value of the feature amount of the voice data in the mora section for each mora section, the voice data of the analysis section is uttered. Change pattern extraction step of extracting a change pattern of the feature amount of the voice that changes for each mora section at the time,
The emotion estimator generation method according to claim 1, further comprising: In the change pattern extracting step, the fundamental frequency of the voice extracted from the voice data is used as the feature amount of the voice, and the average fundamental frequency of the voice in the analysis section and the average fundamental frequency of the voice in the mora section are set for each mora section. In comparison with the above, when the average fundamental frequency of the sound in the mora section is higher than the average fundamental frequency of the sound in the analysis section, High is given, and when it is lower, Low is given, and the sound changes to High and Low for each mora section. The change pattern of the feature amount of
The emotion estimator generation method according to claim 2, wherein. In the change pattern extraction step, the strength of the voice extracted from the voice data is used as the feature quantity of the voice, and the average strength of the voice in the analysis section and the average strength of the voice in the mora section are compared for each mora section. , If the average strength of the sound in the mora section is higher than the average strength of the sound in the analysis section, High is given, and if it is low, Low is given, and the change in the feature amount of the sound changing to High and Low for each mora section. Extract patterns,
The emotion estimator generation method according to claim 2, wherein. In the change pattern extraction step, using the phoneme duration extracted from the voice data as a voice feature amount, the average duration of the phoneme in the analysis section, and the average duration of the phoneme in the mora section, Comparison is made for each mora section. If the average duration of the phonemes in the mora section is longer than the average duration of the phonemes in the analysis section, then High is given, and if it is short, Low is given. A change pattern of the feature amount of the voice that changes to Low is extracted,
The emotion estimator generation method according to claim 2, wherein. In the change pattern extraction step, a change pattern of the voice feature amount is extracted by using at least one of the fundamental frequency of the voice, the voice intensity, and the duration of the phoneme as the voice feature amount.
The emotion estimator generation method according to any one of claims 2 to 5, wherein In the analysis section setting step, the speech data is divided into morphemes, which are the smallest units having a language meaning, and a section of an accent phrase combined with a particle or auxiliary verb uttered after the morpheme is set as the analysis section. To do
The emotion estimator generation method according to any one of claims 1 to 6, characterized in that. In the mora section extraction step, when the voice data is displayed in text, one kana character is set as one mora section, the small kana character is combined with the preceding kana character as one mora section, and the long sound is independent. 1 mora section,
The emotion estimator generation method according to claim 2, wherein. The emotion estimator estimates the emotion of the speaker at the time of utterance to be any one of sadness, boredom, anger, surprise, discouragement, disgust, and joy.
The emotion estimator generation method according to claim 1, wherein A change pattern setting step of setting a change pattern classified into the plurality of emotion classes,
The emotion estimator generation method according to any one of claims 1 to 9, wherein Analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining as a change pattern of the feature amount,
Audio data classified for each change pattern of the feature quantity corresponding to each of the emotion as teacher data, and emotion estimator generating means for generating an emotion estimator for estimating a speaker emotion when the uttered voice ,
Emotion estimator generation device including. An analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states which is a source of teacher data by a computer,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining the change pattern of the feature amount,
The feature quantity of the speech data classified for each change pattern as teacher data, the emotion estimator generating means for generating an emotion estimator for estimating a speaker emotion when the uttered voice corresponding to each of the emotion,
Program to function as. An analysis section setting step for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states to be analyzed,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion A change pattern determination step of determining a change pattern of the feature amount,
For each change pattern of the feature amount corresponding to each emotion, the emotion of the speaker at the time of uttering a voice, which is generated based on the teacher data having the change pattern of the same feature amount as the change pattern of the feature amount , Using an emotion estimator to estimate, an emotion estimation step of estimating the emotion of the speaker when the voice of the analysis section is uttered,
Emotion estimation method including. Analysis section setting means for setting an analysis section for analyzing a feature amount of voice data including a plurality of emotional states to be analyzed,
The pattern of change in the characteristic quantity of audio data included in the analysis interval, based on the change pattern is classified into a plurality of emotion classes, the audio data included in the analysis section corresponding to the class of the plurality of emotion Change pattern determining means for determining as a change pattern of the feature amount,
For each change pattern of the feature amount corresponding to each emotion, the emotion of the speaker at the time of uttering a voice, which is generated based on the teacher data having the change pattern of the same feature amount as the change pattern of the feature amount , Using an emotion estimator for estimating, an emotion estimating means for estimating the emotion of the speaker when the voice of the analysis section is uttered,
Emotion estimation device equipped with.

Images