JP5496863B2 - Emotion estimation apparatus, method, program, and recording medium - Google Patents

Description

  The present invention relates to an emotion estimation apparatus for estimating an anger emotion from target speech data or target text data, a method thereof, a program thereof, and a recording medium thereof.

  Non-Patent Document 1 is known as a conventional technique for estimating an angry emotion from target text data. In Non-Patent Document 1, words related to emotion (hereinafter referred to as “emotion related words”) and emotion scores for emotion related words are stored in the emotion related word DB, and the sentence level is referred to the emotion related word DB. Estimate the emotional score. For example, emotional words such as “children” and “summer vacation” are associated with emotional scores such as “fun” and “positive”, and emotional words such as “war” and “accident” are associated with “anger”. "," Sad "," negative ", etc. are associated with emotional scores.

Hisashi Sugawara, Arena Nevia Roskaya, Mitsuru Ishizuka, “Emotion Extraction from Japanese Text”, Proceedings of the Japanese Society for Artificial Intelligence (CD-ROM), 2009, Volume: 23rd, Page: ROMBUNNO. 3I4-2

  However, since the related art stores the emotion related words and the emotion score for the emotion related words in the emotion related word DB, the following problems occur.

  In order to store a sufficient amount of emotion related words to estimate in the emotion related word DB, enormous human, time and money costs are required. Therefore, practically, an emotion related word DB that records about 300 emotion related words is created. Then, many emotion-related words (hereinafter referred to as “unknown words”) that are not registered in the emotion-related word DB are included in the target text data, and the emotion score cannot be estimated frequently.

  Moreover, even if the same emotion-related word is used, the related emotions may differ depending on the context and usage (hereinafter referred to as “context etc.”). For example, in the sentence “Your attitude is like a child,” “kid” is related to “negative” emotions, but it is registered in the emotion related word DB as emotion related words related to “positive” emotions. It may have been the cause of estimating the wrong emotion.

  Therefore, an emotion estimation apparatus, method, program thereof, and recording medium thereof that can estimate emotion from target speech data or target text data without using a database that records semantic information of words such as emotion related word DB There is a problem of providing. The problem of not being able to estimate emotions from the target text data due to the presence of unknown words due to a configuration that does not use a database (for example, emotion-related word DB) that records the semantic information of words, and erroneously estimating emotions due to context, etc. The problem of doing no longer occurs.

  In order to solve the above problem, the present invention uses at least one of the morphological analysis result and the syntax analysis result of the target text data to express how much the semantic content is clearly indicated in the target text data. Estimate gender features and use an identifier that has been learned in advance from the explicit features of learning text data with a teacher signal to indicate whether anger emotions are expressed or not. Whether the emotion is expressed is identified from the explicit feature quantity of the target text data estimated by the explicit estimation unit.

  The emotion estimation apparatus according to the present invention has an effect of being able to estimate an angry emotion expressed in the target speech data or the target text data without being affected by unknown words, contexts, and the like.

The block diagram which shows the function structural example of the emotion estimation apparatus 100. FIG. The figure which shows the example of the processing flow of the emotion estimation apparatus 100. FIG. The figure which shows the example of a morphological analysis result. The figure which shows the example of a parsing result. The figure which shows the example of the parsing result used as the input to an abbreviation case estimation part. The figure which shows the example of the morphological analysis result used as the input to a replacement word estimation part or an explicit word detection part. The figure which shows the example of data of an explicit word list memory | storage part. The block diagram which shows the function structural example of the emotion estimation apparatus 100 which produces | generates a discriminator. The figure which shows the example of the processing flow of the emotion estimation apparatus 100 which produces | generates a discriminator. The block diagram which shows the function structural example of the emotion estimation apparatuses 200 and 300. FIG. The figure which shows the example of the processing flow of the emotion estimation apparatuses 200 and 300. FIG. The block diagram which shows the function structural example of the emotion estimation apparatuses 200 and 300 which produce | generate a discriminator. The figure which shows the example of the processing flow of the emotion estimation apparatuses 200 and 300 which produce | generate a discriminator. The block diagram which shows the function structural example of the monitoring system 400. FIG. The figure which shows the example of the output data of the monitoring system. The block diagram which shows the function structural example of the monitoring system 500. FIG. The figure which shows the example of the output data of the monitoring system.

[Points of Invention]
In the present invention, in order to estimate the feeling of anger expressed in the target speech data or the target text data, attention is paid to the explicitness of the sentence, in particular, the explicitness of the utterance contents in the general conversation situation. Note that the clarification of the utterance content refers to how much omission of the subject or object or substitution with an instruction word such as “it” or “this” occurs in the utterance. The case where these omissions and substitutions have not occurred is defined as a state where the utterance content is highly explicit. A case where many of these omissions and substitutions occur is defined as a state where the utterance content is not clearly specified.

  In order to show that there is a difference in clarification between angry utterance contents and non-angry utterance contents, the following analysis results will be described.

  The labels of anger utterance (utterance number 3312) and normal utterance (utterance number 3367) are manually assigned to the total number of utterances 6679 in a call center. The discriminator is learned by SVM (support vector machine) using the labeled utterance. A word list composed of 256 words obtained by manually removing service names, personal names, numbers, business terms, etc. from words (unigrams) that appear biased to angry utterances at a significance level of 1% by the chi-square test is generated. At this time, a normalized value obtained by dividing the number of appearances of the word included in the utterance by the total number of morphemes included in the utterance is used as the feature.

なお、ランダム抽出した場合の適合率は５０．５％であり、怒っている発話内容と怒っていない発話内容とで言語的特徴があることがわかる。なお、怒り発話に偏って出現した単語としては、一人称の「私」や「わたしら」、二人称の「お宅」等がある。 Using the word list as a parameter, the analysis target utterance is input to the learned classifier, and the analysis target utterance is identified as an angry utterance and a normal utterance, or an angry utterance and an angry utterance. Call center utterances include customer utterances and operator utterances, but categorize customer utterances into angry utterances and normal utterances, and utter utterances that are angry and angry. And classify. The angry utterance means the utterance of the operator immediately after the customer's angry utterance, and the other utterances are uttered not angry. The analysis results are shown below.

In addition, the relevance rate in the case of random extraction is 50.5%, and it can be seen that there is a linguistic feature between angry utterance content and non-angry utterance content. Note that words that appear biased in anger utterance include first-person “I” and “we”, second-person “home”, and the like.

When a speaker whose mother tongue is Japanese speaks with people, words such as “you” and “I am” are often omitted. In addition, for example, when pointing to something, substitution to a directive word often occurs or is omitted. This is a language that allows Japanese language to be as ambiguous as possible, and is caused by Japanese social culture that is vague in interpersonal communication. However, when the speaker's emotional state is "anger", he tries to eliminate the ambiguity and make the target of anger clearer. For example, the second person noun such as “you” is rare in communication during normal times, but the word “you” appears frequently when angry. Similarly, words that were omitted and reported during normal times are often revealed when angry. For example, when a child spills soy sauce, the following difference appears between the normal speech and the angry speech as to the mother's child.
Normal utterance: “Did you spill?”
Anger utterance: “Did you spill soy sauce?”
The present invention focuses on this phenomenon. As the clarification of the utterance content, “how much omission or replacement with the instruction word occurs during the utterance” is calculated, and when the utterance content clarification is high, it is estimated that the speaker is angry.

  Hereinafter, embodiments of the present invention will be described in detail.

<Emotion estimation apparatus 100>
The emotion estimation apparatus 100 according to the first embodiment will be described with reference to FIGS. 1 and 2. The emotion estimation device 100 includes a text analysis unit 130, an explicitness estimation unit 140, a discriminator generation unit 150, and an emotion identification unit 160. With such a configuration, the emotion estimation apparatus 100 estimates an angry emotion from the target text data. Note that a unit for estimating an emotion in the target text data is called an emotion estimation unit. The emotion estimation unit may be a single sentence, a plurality of sentences, or a specific section in the sentence. Details of each part will be described below.

<Text analysis unit 130>
The text analysis unit 130 analyzes the target text data and obtains a morpheme analysis result obtained from the target text data and a syntax analysis result obtained by analyzing the dependency relationship in the target text data based on the morpheme analysis result ( s130). For example, the text analysis unit 130 includes a morpheme analysis unit 131 and a syntax analysis unit 133. Details of each part will be described below.

(Morphological analyzer 131)
The morpheme analysis unit 131 receives the target text data, performs morpheme analysis on the target text data, obtains a morpheme analysis result (s131), and obtains it as a syntax analysis unit 133, a replacement word estimation unit 143, and an explicit word detection unit 144. Output to. Note that a morpheme is the smallest unit that has linguistic meaning. A morpheme analysis is information about grammar knowledge (a collection of grammar rules) or a dictionary (a word list with information such as parts of speech) in the target language. This is a task that uses sentences as a source and divides sentences written in natural language into morpheme strings and discriminates each part of speech. It is assumed that knowledge of the target language grammar (a collection of grammar rules) and a dictionary (a word list with information such as parts of speech) are stored in a storage unit (not shown). As a morphological analysis technique, a conventional technique (for example, “Makoto Nagao (ed.),“ Natural Language Processing ”, Iwanami Course Software Science, Vol. 15, Iwanami Shoten, 1996 (hereinafter referred to as“ Reference Document 1 ”)) is described. Technology).

  For example, the morpheme analysis unit 131 divides the target text data into morpheme units, determines the part of speech of each divided morpheme, and outputs the target text data with the part of speech added to each morpheme as a morpheme analysis result. For example, if the target text data “I saw a girl” is subjected to morphological analysis, the morphological analysis result shown in FIG. 3 is obtained. In FIG. 3, “/” represents a morpheme break.

(Syntax analyzer 133)
The syntax analysis unit 133 receives the morpheme analysis result as input, parses the morpheme analysis result, obtains the syntax analysis result (s133), and outputs the result to the abbreviation estimation unit 141. For example, the syntax analysis unit 133 obtains a clause from the morphological analysis result, further analyzes which clause is the main part, which clause is the predicate, the dependency relation of the clause, and the analysis result is the morphological analysis. The result is added to the result as the result of parsing. For example, when the morphological analysis result shown in FIG. 3 is parsed, the parse analysis result shown in FIG. 4 (the morpheme analysis result with the dependency relationship and the main part / predicate etc. added to each phrase) is obtained. As a syntax analysis technique, a conventional technique (for example, a conventional technique described in Reference 1) can be used.

<Explicitness estimation unit 140>
The explicitness estimation unit 140 uses the morphological analysis result obtained from the target text data and the syntactic analysis result obtained by analyzing the dependency relationship in the target text data based on the morphological analysis result to determine how much in the target text data. An explicit feature amount indicating whether the semantic content is specified is estimated (s140). For example, the explicitness estimation unit 140 includes an omitted case estimation unit 141, a replacement word estimation unit 143, an explicit word detection unit 144, an explicit word list storage unit 144a, and an explicit feature quantity calculation unit 145. Details of each part will be described below.

(Omitted case estimation unit 141)
The abbreviation estimation unit 141 estimates the abbreviation that should be originally present in the target text data as an abbreviation using the syntax analysis result (s141), and clearly indicates the type of abbreviation and the number of abbreviations of each abbreviation. It outputs to the sex feature amount calculation unit 145.

  As a method for estimating the omitted case, for example, there is a method using a case frame dictionary. For example, [Daisuke Kawahara, Ikuo Kurohashi, "Large-scale evaluation of omission analysis based on automatically constructed case frame dictionary", The 9th Annual Conference of the Language Processing Society, pp.589-592, 2003. 2))] can be estimated using a case frame dictionary. Also, instead of an automatically constructed case frame dictionary, an existing case frame dictionary, for example, a case frame dictionary such as the IPAL verb dictionary [Technology Center for Information Processing Promotion Corporation, Japanese Basic Verb Dictionary for Computers IPAL, 1987] is used. An abbreviated case may be estimated. At this time, the case frame dictionary is stored in advance in a storage unit (not shown).

  A case is a sign indicating a semantic relationship between words in a sentence, especially a verb and a noun (or noun phrase), and a case frame is a case structure of a verb. The case frame dictionary is a database in which case frames are described for individual verbs. There are two types of cases: surface cases that are determined by the surface layer (such as “ga”, “wo”, and “d”) in Japanese, and deep cases that represent the true case that are not determined solely by the surface, When defining the case that the verb can take in the deep case, (1) Action case (Agent), (2) Experiencer case (Experencer), (3) Instrument case (Object), (4) Object case (Object) ), (5) Source case (6), (6) Target case (Goal), (7) Location case (Location), (8) Time case (Time), etc. are used.

  Furthermore, the case frame dictionary defines an optional case that may be taken as an indispensable case that a verb must take. For example, in the case of the verb “see”, “action case” and “target case” are indispensable cases, and “location case” and “tool case” are arbitrary cases. The abbreviation estimation means processing for estimating whether or not an essential case for a certain verb is omitted with reference to a case frame dictionary. When the abbreviated case is estimated for the parsing result shown in FIG. 5, the “target case” that is an essential case of “see” and the “motion main case” that is an essential case of “surprise” are omitted. The type and number of abbreviations (“target case”, 1) and (“operation main case”, 1) are output as abbreviated case estimation results.

  In the case frame dictionary, other elements, for example, semantic restrictions on the case may be defined.

(Replacement word estimation unit 143)
Using the morphological analysis result, the replacement word estimation unit 143 estimates a word that exists in the target text data in place of the original word as a replacement word (s143), and the type of replacement word and the number of occurrences of each replacement word Is output to the explicit feature quantity calculation unit 145. For example, an instruction word is used as a word used after replacement (that is, a “replacement word”). An instruction word is an expression that indicates an element in the field or a contextual element, for example, “this, it, that”, and the like, and is used as a substitute for a word that should originally exist.

  For example, when the replacement word estimation unit 143 receives the morphological analysis result shown in FIG. 6 and performs the replacement word estimation, the replacement word is the instruction word “this”, and therefore the type and number of replacement words (“this”). 1) is output as a replacement word estimation result.

(Explicit word detection unit 144 and explicit word list storage unit 144a)
The explicit word list storage unit 144a stores in advance words that are easily omitted in normal speech. As explained in the point of the invention, there are words that appear biased during anger utterances. For example, as described above, in the normal utterance, the first person “I” and “we”, the second person “home”, etc. are easily omitted, but in the angry utterance, there is a tendency that it is difficult to omit. Words having these tendencies are stored in advance in the explicit word list storage unit 144a as explicit words (see FIG. 7).

  The explicit word detection unit 144 receives the morpheme analysis result as an input, uses this to refer to the explicit word list storage unit 144a, detects an explicit word existing in the target text data (s144), The number is output as an explicit word detection result.

  For example, when the explicit word detection unit 144 receives the morphological analysis result shown in FIG. 6 and performs explicit word detection using the explicit word list storage unit 144a shown in FIG. 7, the word “I” is an explicit word. The type and number of words (“I”, 1) are output as an explicit word detection result.

(How to create an explicit word list)
The explicit words stored in the explicit word list storage unit 144a may be limited to pronouns (for example, personal pronouns “you” and “I”). This is because pronouns and the like tend to be omitted in normal utterances and are difficult to omit in angry utterances, and this tendency is not influenced by domains or the like. A domain is a medium (newspaper, magazine, call center speech recognition result, TV speech recognition result, etc.) or field (sports article, political article, economic article) from which learning text data or target text data is acquired. , Voice recognition results of calls at insurance company call centers, voice recognition results of calls at communication company call centers, and the like. In this case, by targeting only pronouns or the like, it is not necessary to cover all words, and the cost of creating an explicit word list can be reduced. It also reduces the cost of creating lists by domain.

  The explicit word stored in the explicit word list storage unit 144a may be determined by the method described in the point of the invention. First, anger utterances and normal utterance labels are manually attached to the sample text data. Next, out of words (unigrams) that are biased toward angry utterances with a significance level of 1% in the chi-square test, those that are obtained by manually removing service names, names, numbers, business terms, etc. are designated as explicit words. This list of explicit words is stored in advance in the explicit word list storage unit 144a. The significance level of the χ square test may be set as appropriate by obtaining an appropriate value in advance through experiments or the like.

  Also, the explicit word list storage unit 144a tends to be omitted for normal utterances and hard to be omitted for angry utterances for each case type (that is, for each “actor main”, “target case”, etc.). The word may be stored as an explicit word. In that case, when the above-mentioned explicit word list is created, it is classified in advance for each case type. The explicit word detection unit 144 uses the syntax analysis result to refer to the explicit word list storage unit 144a to detect a word that matches the word and the type of case of the word as an explicit word.

(Explicit feature quantity calculation unit 145)
The explicit feature quantity calculation unit 145 calculates an explicit feature quantity using the omitted case estimation result, the replacement word estimation result, and the explicit word detection result (s145), and outputs the explicit feature quantity to the emotion identification unit 160. For example, the explicit feature quantity calculation unit 145 calculates, as an explicit feature quantity, one of the following values obtained by using the abbreviation case estimation result, the replacement word estimation result, and the explicit word detection result: To do.
(1) Number of omissions for each abbreviation (2) Presence / absence of omission for each abbreviation ("0""1" information)
(3) Total number of omissions of all abbreviations included in the emotion estimation unit instead of each abbreviation (4) Presence / absence of abbreviations in the emotion estimation unit (“0” “1” information)
(5) Number of appearances for each replacement word (6) Presence / absence of each replacement word (“0” “1” information)
(7) The total number of appearances of all replacement words included in the emotion estimation unit instead of each replacement word (8) Presence / absence of the replacement word in the emotion estimation unit (“0” “1” information)
(9) Number of appearances for each explicit word (10) Presence / absence of each explicit word (“0” and “1” information)
(11) The total number of appearances of all explicit words included in the emotion estimation unit, not every explicit word (12) Presence / absence of the explicit word in the emotion estimation unit (“0” “1” information)
When any one of (1), (5), and (9) is used as the explicit feature amount, the abbreviation estimation result, the replacement word estimation result, and the explicit word detection result On the other hand, since no special processing is required, the explicit feature quantity calculation unit 145 is not provided, and the abbreviation estimation unit 141, the replacement word estimation unit 143, and the explicit word detection unit 144 respectively include an abbreviation estimation result and a replacement. The word estimation result and the explicit word detection result may be directly output to the emotion identification unit 160 as the explicit feature amount.

  Increasing the number of elements of the explicit feature value increases the accuracy of identification in the emotion identification unit 160 described later, but increases the amount of calculation. Therefore, in order to select an appropriate element of explicit feature quantity, sample text data may be prepared according to target text data, and the element of explicit feature quantity may be determined in advance by experiments or the like.

<Emotion identification unit 160>
The emotion discriminating unit 160 receives the explicit feature quantity as input, and uses the discriminator learned in advance in the discriminator generating unit 150 to be described later to determine whether or not an angry emotion is expressed in the target text data. Identification is performed from the explicit feature quantity of the target text data estimated by the explicit estimation unit 140 (s160).

  The emotion discriminating unit 160 may output the discrimination result (“anger” or “normal”) of the discriminator as it is as the estimation result of the emotion estimation device 100, or the discriminator also outputs the likelihood for the discrimination result. In this case, the estimation result may be output as follows. (A) When the identification result is “anger” and the likelihood is a first threshold (for example, 0.7) or more, the estimation result is “anger”, and (B) the identification result is “anger” If the likelihood is less than the first threshold and the second threshold (for example, 0.3) or more, the estimation result is impossible to estimate, and (C) otherwise (that is, the identification result is “normal”) When the likelihood is less than the second threshold), the estimation result is “normal”.

<Identifier generation unit 150>
The discriminator generation unit 150 generates a discriminator using the explicit feature amount of the learning text data with the teacher signal (s150 in FIG. 9). The teacher signal is information indicating whether or not an angry emotion is expressed in corresponding learning text data. A person looks at each learning text data to determine whether or not an angry emotion is expressed, and adds a teacher signal to each learning text data.

  There is a machine learning method as a method for generating a discriminator. For example, various learning algorithms for machine learning can be adopted, and a supervised learning linear discriminant method, SVM, neural network, or the like can be used.

<Calculation method of explicit feature quantity of text data for learning>
A method for calculating the explicit feature value of the text data for learning used in the classifier generation unit 150 will be described with reference to FIGS. 8 and 9.

  The text analysis unit 130 analyzes the learning text data input from the learning text corpus 90, the morpheme analysis result obtained from the learning text data, and the dependency in the learning text data based on the morpheme analysis result. A syntax analysis result obtained by analyzing the relationship is obtained (s130-2).

  The explicitness estimation unit 140 uses the morpheme analysis result obtained from the learning text data and the syntax analysis result obtained by analyzing the dependency relationship in the learning text data based on the morpheme analysis result, to obtain the learning text data. Then, an explicit feature amount indicating how much the semantic content is clearly shown is estimated (s140-2).

  That is, the discriminator generation unit 150 performs the same processing in the text analysis unit 130 and the explicitness estimation unit 140 based on the learning text data instead of the target text data without providing a special configuration. The explicit feature quantity of the text data for learning to be used can be acquired.

<Effect>
With such a configuration, an angry emotion can be estimated without using a database (for example, an emotion related word DB) in which word semantic information is recorded. Therefore, it is possible to reduce the creation cost of the emotion related word DB and the like, and it is possible to robustly estimate the anger emotion expressed in the target text data without being influenced by the unknown word and the context.

[Modification]
When morphological analysis, syntax analysis, etc. are completed in advance on the target text data in another device or the like, and the morphological analysis result and syntax analysis result of the target text data are input to the emotion estimation device 100, The emotion estimation device 100 may not include the text analysis unit 130.

  The explicitness estimation unit 140 may include at least one of the abbreviation estimation unit 141, the replacement word estimation unit 143, and the explicit word detection unit 144 as necessary. For example, when calculating any one of (1) to (4) described in the explicit feature value calculation unit 145 or a combination of (1) to (4) as the explicit feature value, at least omitted. What is necessary is just to provide the case estimation part 141. FIG. If any one of (5) to (8) or a combination of (5) to (8) is calculated as the explicit feature value, at least the replacement word estimation unit 143 may be provided. When calculating any one of (9) to (12) or a combination of (9) to (12) as the explicit feature amount, at least the explicit word detecting unit 144 may be provided. If the abbreviation estimation unit 141 is not provided, the syntax analysis result is not required. Therefore, the syntax analysis unit 133 may not be provided. If the explicit word detection unit 144 is not provided, an explicit word list storage unit is provided. 144a may not be provided.

  The emotion estimation device 100 does not necessarily include the discriminator generation unit 150. For example, when a discriminator learned and generated with a certain emotion estimation device is used with another emotion estimation device, only a certain emotion estimation device has a discriminator generation unit.

  Emotion estimation device 100 may receive target audio data as input. In that case, a speech recognition unit (not shown) is provided in the preceding stage of the text analysis unit 130. Furthermore, when a speech recognition unit (not shown) performs morphological analysis and syntax analysis in the course of speech recognition processing, the morphological analysis result and syntax analysis result corresponding to the target text data obtained as a result of speech recognition are clearly displayed. It is good also as a structure used by the estimation part 140. FIG. Also in this case, the emotion estimation apparatus 100 may not include the text analysis unit 130.

<Emotion estimation device 200>
An emotion estimation apparatus 200 according to the second embodiment will be described with reference to FIGS. 10 and 11. Only parts different from the first embodiment will be described. The emotion estimation device 200 includes a voice recognition unit 210, a text analysis unit 130, an explicitness estimation unit 240, a discriminator generation unit 150, and an emotion identification unit 160. The processing contents of the text analysis unit 130, the classifier generation unit 150, and the emotion identification unit 160 are the same as those in the first embodiment. However, the recognition reliability described later is added to the target text data handled by the text analysis unit 130, and the text analysis unit 130 outputs a syntax analysis result and a morphological analysis result with the recognition reliability (see FIG. 10). Further, the explicit feature quantity processed in the discriminator generation unit 150 and the emotion discrimination unit 160 is a value obtained using this recognition reliability.

<Voice recognition unit 210>
The voice recognition unit 210 receives the target voice data, performs voice recognition processing on the target voice data, and converts it into target text data. The speech recognition unit 210 obtains the target text data obtained from the target speech data and the recognition reliability indicating the reliability of the recognition result for each word included in the target text data (s210), with the recognition reliability. Is output to the text analysis unit 130. Note that the recognition reliability is a value indicating the reliability of the likelihood of the recognition result. If the recognition reliability is high, it is presumed that the recognition result is correct, and if it is low, the recognition result is presumed to be incorrect. In addition, as speech recognition technology, description is made on the prior art (for example, [Sadaaki Furui, “Acoustic / Speech Engineering (Introduction to Electronic / Information Engineering)”), Modern Science Co., 1992 (hereinafter referred to as “Reference 3”)]. Can be used. When the target voice data is recorded in stereo for each speaker, voice recognition is easier than in the case of monaural recording. In the case of monaural recording, it is used in combination with a means for identifying speech for each speaker. As the speaker identification technique, a conventional technique (for example, a conventional technique described in Reference 3) can be used. For example, there is a method of using a GMM (Gaussian Mixture Model) with an audio spectrum as a feature quantity.

  When performing morpheme analysis and syntax analysis in the speech recognition process, the explicitness estimation unit 240 may use the morpheme analysis result and syntax analysis result corresponding to the target text data obtained as a result of speech recognition. Good. Also in this case, the emotion estimation apparatus 200 may not include the text analysis unit 130.

<Explicitness estimation unit 240>
The explicitness estimation unit 240 uses an at least one of a morphological analysis result with recognition reliability and a syntactic analysis result with recognition reliability to express how much the semantic content is clearly specified in the target text data. The sex characteristic amount is estimated (s240). The explicitness estimation unit 240 includes an abbreviated case estimation unit 241, a replacement word estimation unit 243, an explicit word detection unit 244, and an explicit feature quantity calculation unit 245. Details will be described below.

(Omitted case estimation unit 241)
The omitted case estimation unit 241 receives the parsing result with the recognition reliability as an input, and estimates the omitted case that should originally exist in the target text data as the omitted case (s141). The result is output to the explicit feature value calculation unit 145.

  For example, the abbreviation estimation unit 241 can estimate the abbreviation robustly even for a speech recognition result including an error by combining the recognition reliability when estimating the abbreviation as follows. It becomes possible. For example, in the target text data of FIG. 5, the recognition reliability of the verb “see” is “0.8”, and the recognition reliability of the verb “surprise” is “0.2”.

(1) The abbreviation estimation unit 241 outputs, as an abbreviation estimation result, the type of abbreviation for a verb whose recognition reliability is equal to or greater than a certain threshold and the number of abbreviations of that abbreviation. The threshold value is determined in advance. If the threshold value is “0.5”, only the abbreviations (target case) for the verb “see” whose recognition reliability is “0.5” or more are totaled, and the replacement word estimation result (“target case”, 1) is output.
(2) The replacement word estimation unit 243 outputs the sum of the abbreviated case type and the verb recognition reliability value for the abbreviated case as a replacement word estimation result. In this case, (“target case”, 0.8) and (“operation main case”, 0.2) are output.

  Since the recognition reliability is not absolute, just because the recognition reliability is low does not necessarily mean that it is wrong. Rather than simply summing up with threshold values as in (1), by summing up as in (2), abbreviations for verbs with low recognition reliability can be reflected in the estimation results.

(Replacement word estimation unit 243)
The replacement word estimation unit 243 uses the morphological analysis result with the recognition reliability to estimate a word that exists in the target text data instead of the word that should be present as a replacement word (s243), and clearly indicates the replacement word estimation result. It outputs to the sex feature amount calculation unit 145.

  For example, the replacement word estimation unit 243 can robustly estimate the replacement word even for the speech recognition result including the error by combining the recognition reliability when estimating the replacement word as follows. It becomes possible. For example, assuming that the replacement word “it” appears twice in the target text data, the recognition reliability of the first “it” is “0.8”, and the recognition reliability of the second “it” Is “0.2”.

(1) The replacement word estimation unit 243 outputs, as a replacement word estimation result, the type of replacement word whose recognition reliability is equal to or higher than a certain threshold and the number of appearances of the replacement word. The threshold value is determined in advance. If the threshold is “0.5”, only the first “it” whose recognition reliability is “0.5” or more is totaled, and (“it”, one) is output as the replacement word estimation result. To do.
(2) The replacement word estimation unit 243 outputs the sum of the type of replacement word and the recognition reliability value of the replacement word as a replacement word estimation result. In this case, (“it”, 1 (= 0.8 + 0.2)) is output.

Since the recognition reliability is not absolute, just because the recognition reliability is low does not necessarily mean that it is wrong. Rather than simply summing up with threshold values as in (1), replacement words with low recognition reliability can be reflected in the estimation results by summing up as in (2).
(Explicit word detection unit 244)
The explicit word detection unit 244 refers to the explicit word list storage unit 144a using the morphological analysis result with the recognition reliability, detects an explicit word existing in the target text data (s244), and displays the explicit word detection result. This is output to the explicit feature quantity calculation unit 145.

  The explicit word detection unit 244 can detect an explicit word robustly even for a speech recognition result including an error by combining recognition reliability when detecting an explicit word, for example, as follows. It becomes possible. For example, if the explicit word “you” appears twice in the target text data, the recognition reliability of the first “you” is “0.8” and the recognition reliability of the second “you” Is "0.2", and the explicit word list storage unit 114a stores the explicit word list shown in FIG.

(1) The explicit word detection unit 244 outputs, as an explicit word detection result, the type of explicit word whose recognition reliability is equal to or higher than a certain threshold and the number of appearances of the explicit word. The threshold value is determined in advance. If the threshold is set to “0.5”, only the first “you” whose recognition reliability is “0.5” or higher is aggregated, and (“you”, 1 item) is output as the explicit word detection result. To do.
(2) The explicit word detection unit 244 outputs the sum of the type of the explicit word and the recognition reliability value of the explicit word as an explicit word detection result. In this case, (“you”, 1 (= 0.8 + 0.2)) is output.

  Since the recognition reliability is not absolute, just because the recognition reliability is low does not necessarily mean that it is wrong. Rather than simply summing up with threshold values as in (1), by summing up as in (2), explicit words with low recognition reliability can be reflected in the detection results.

(Explicit feature calculation unit 245)
The explicit feature quantity calculation unit 245 calculates an explicit feature quantity using the omitted case estimation result, the replacement word estimation result, and the explicit word detection result (s245), and outputs the explicit feature quantity to the emotion identification unit 160. For example, the explicit feature quantity calculation unit 245 calculates, as an explicit feature quantity, one of the following values obtained by using the abbreviation case estimation result, the replacement word estimation result, and the explicit word detection result: To do.

(1 ′) Number of omissions for each abbreviation, or sum of recognition reliability values of verbs corresponding to each abbreviation (2) Presence / absence of omission for each abbreviation (“0” “1” information)
(3 ′) The total number of omissions of all abbreviations included in the emotion estimation unit, not the abbreviations, or the recognition reliability value of the verb corresponding to all abbreviations included in the emotion estimation unit Sum (4) Presence or absence of abbreviation in emotion estimation unit ("0""1" information)
(5 ′) Number of occurrences for each replacement word or sum of recognition reliability values for each replacement word (6) Presence / absence of each replacement word (“0” “1” information)
(7 ′) The total number of appearances of all replacement words included in the emotion estimation unit, or the sum of recognition reliability values of all replacement words included in the emotion estimation unit, instead of each replacement word (8) Presence / absence of replacement word in emotion estimation unit ("0""1" information)
(9 ') Number of appearances for each explicit word or the sum of recognition reliability values for each explicit word (10) Presence / absence of each explicit word ("0""1" information)
(11 ′) The total number of appearances of all the explicit words included in the emotion estimation unit, or the sum of the recognition reliability values of all the explicit words included in the emotion estimation unit, not every explicit word (12) Presence / absence of explicit word in emotion estimation unit ("0""1" information)
Note that (1 ′), (3 ′), (5 ′), (7 ′), (9 ′), and (11 ′) are different from the explicit feature value calculation unit 145.

<Calculation method of explicit feature quantity of text data for learning>
A method for calculating the explicit feature amount of the text data for learning used in the classifier generation unit 150 will be described with reference to FIGS. 12 and 13.

  The speech recognition unit 210 performs speech recognition processing on the learning speech data input from the learning speech corpus 91 and converts it into learning text data. The speech recognition unit 210 obtains learning text data obtained from the learning speech data and a recognition reliability indicating the reliability of the recognition result for each word included in the learning text data (s210-2). The learning text data with recognition reliability is output to the text analysis unit 130.

  The text analysis unit 130 analyzes the learning text data with recognition reliability input from the speech recognition unit 210 and learns based on the morpheme analysis result with recognition reliability obtained from the learning text data and the morpheme analysis result. The syntactic analysis result with the recognition reliability obtained by analyzing the dependency relation in the text data for use is obtained (s130-2).

  The explicitness estimation unit 140 includes a morpheme analysis result with recognition reliability obtained from the text data for learning, and a syntax analysis result with recognition reliability obtained by analyzing the dependency relationship in the text data for learning based on the morpheme analysis result, Is used to estimate an explicit feature amount indicating how much the semantic content is clearly specified in the learning text data (s240-2).

  That is, even if no special configuration is provided, identification is performed by performing the same processing in the speech recognition unit 210, the text analysis unit 130, and the explicitness estimation unit 240 based on the learning speech data instead of the target speech data. The explicit feature amount of the text data for learning used in the generator generator 150 can be acquired.

<Effect>
With such a configuration, the same effect as in the first embodiment can be obtained. Further, by using the recognition reliability, it is possible to robustly estimate the explicitness even for the speech recognition result including an error.

  It is generally known that prosodic information is effective in identifying emotions. Therefore, when the input is voice data, the emotion estimation apparatus 200 according to the second embodiment may be combined with prosodic information such as voice pitch and voice volume to estimate emotion.

  In the third embodiment, the pitch, power, and the like of the target audio data are calculated, and the average value, maximum / minimum value, variance, and the like thereof, and the dynamic feature value Δ and the value of ΔΔ are used as prosodic feature values. Use. These prosodic feature values and explicit feature values are used in combination to identify emotions in the emotion identification unit. With such a configuration, the estimation accuracy can be further increased.

<Emotion estimation device 300>
An emotion estimation apparatus 300 according to the third embodiment will be described with reference to FIGS. 10 and 11. Only parts different from the second embodiment will be described. The emotion estimation device 300 includes a voice recognition unit 210, a text analysis unit 130, an explicitness estimation unit 240, a discriminator generation unit 350, and an emotion identification unit 360, and also in broken lines in FIGS. A prosodic feature quantity calculation unit 320. The processing contents of the speech recognition unit 210, the text analysis unit 130, and the explicitness estimation unit 240 are the same as those in the second embodiment.

<Prosodic feature amount calculation unit 320>
The prosodic feature value calculation unit 320 calculates the prosodic feature value using the target speech data (s320 indicated by a broken line in FIG. 11). As a technique for calculating the prosodic feature value, a conventional technique (for example, a conventional technique described in Reference 3) can be used. Pitch (voice pitch), power (voice volume), etc. are used as prosodic features. For example, one of the following values or a combination thereof is calculated as the prosodic feature quantity.
(A): Average value of pitch (B): Maximum value of pitch (C): Dispersion value of pitch (D): Average value of power (E): Maximum value of power (F): Dispersion value of power (G ): Any increment Δ of (A) to (F)
(H): Increment ΔΔ of (G)
Note that by using (G) and (H), the magnitude of fluctuation such as a steep rise can be captured. As with the explicit feature value, increasing the number of elements of the prosodic feature value increases the accuracy of discrimination in the emotion discriminator 360 described later, but also increases the calculation amount. Therefore, elements of prosodic feature values may be determined in advance by experiments or the like.

<Emotion identification unit 360>
The emotion discriminating unit 360 uses the discriminator previously learned from the prosodic feature quantity of the learning speech data corresponding to the learning text data in addition to the explicit feature quantity of the learning text data, Whether or not an angry emotion is expressed is discriminated from the explicit feature quantity estimated by the explicitness estimation unit 240 and the prosodic feature quantity estimated by the prosodic feature quantity calculation unit 320 (s360). The result is output as the estimation result of emotion estimation apparatus 300.

<Identifier generation unit 350>
The discriminator generation unit 350 learns a discriminator from prosodic feature quantities of learning speech data corresponding to learning text data with a teacher signal in addition to the explicit feature quantity of learning text data with a teacher signal, It is generated (s350 in FIG. 13). The teacher signal is information indicating whether or not an angry emotion is expressed in the corresponding learning text data and learning voice data. In the third embodiment, in order to consider prosodic feature values in addition to explicit feature values, a person listens to each learning speech data, and a person looks at learning text data to learn text data and learning data. It is comprehensively determined whether or not an angry emotion is expressed in the voice data, and a teacher signal is added to each learning text data.

<Calculation method of explicit feature quantity of text data for learning>
A method of calculating prosodic feature quantities of learning speech data and explicit feature quantities of learning text data used in the discriminator generation unit 350 will be described with reference to FIGS. 12 and 13.

  The prosodic feature value calculation unit 320 calculates the prosodic feature value using the learning speech data input from the learning speech corpus 91 (s320-2 indicated by a broken line in FIG. 12), and generates a discriminator. To the unit 350.

  By the same processing (s210-2 to s240-2) as in the second embodiment, the explicit feature amount of the text data for learning is estimated and output to the discriminator generating unit 350.

  That is, even if no special configuration is provided, the same applies to the prosodic feature value calculation unit 320, the speech recognition unit 210, the text analysis unit 130, and the explicitness estimation unit 140 based on the learning speech data, not the target speech data. By performing this process, it is possible to acquire the prosodic feature quantity of the learning speech data used by the discriminator generation unit 350 and the explicit feature quantity of the learning text data.

<Effect>
With such a configuration, prosodic feature quantities can be taken into account, and the discrimination performance can be further improved.

[Modification]
In the third embodiment, the prosodic feature quantity calculation unit 320 calculates the prosodic feature quantity of the target speech data. However, if the prosodic feature quantity can be calculated in the process of speech recognition processing, the prosodic feature quantity is separately provided. The feature amount calculation unit 320 may not be provided and may be part of the voice recognition unit 210.

  In the third embodiment, the speech recognition unit 210 outputs learning text data with recognition reliability, but learning text data to which no recognition reliability is added may be used. In that case, the explicitness estimation unit 140 of the first embodiment may be used instead of the explicitness estimation unit 240.

  In recent years, there has been a lot of movement to obtain useful information for companies based on raw voices such as requests and dissatisfaction from customers gathering at call centers held by companies. In addition, call centers have started to be regarded as important as corporate functions, and companies are also making efforts to improve the quality of service at call centers in order to improve the image customers have about companies. Under such circumstances, a technique for automatically finding a complaint call in which a customer is angry is desired more than ever. Therefore, a monitoring system using the emotion estimation apparatus 200 will be described.

<Monitoring system 400>
As shown in FIG. 14, the monitoring system 400 includes the emotion estimation apparatus 200 described above, an input unit 480, and an output unit 490. The monitoring system 400 is installed in a call center or the like, and monitors incoming calls from the customer to the call center and outgoing calls from the call center to the customer to detect a complaint call.

Emotion estimation apparatus 200 receives target speech data C ₁ , C ₂ ,..., C _N (where N is an integer equal to or greater than 1) via input unit 480. The input unit 480 is, for example, a voice input terminal. The input unit 480 is connected to the telephones 1 ₁ , 1 ₂ ,..., 1 _N installed in the call center, and the monitoring system 400 receives the call contents of the customer and the operator as target voice data in real time. be able to.

The emotion estimation apparatus 200 divides the target speech data into predetermined emotion estimation units, estimates emotions for each emotion estimation unit, and sets telephone identifiers P ₁ , P ₂ ,..., P _N and their estimation results E ₁ , E _2, ..., and outputs the _{E N} at any time, to the output unit 490.

  The output unit 490 is, for example, a display. The output unit 490 displays the telephone identifier and the estimation result (see FIG. 15).

<Effect>
With such a configuration, the call center manager or the like can monitor the operator's response status in real time, and can quickly detect and respond to the occurrence of a complaint.

[Modification 1]
<Monitoring system 500>
As shown in FIG. 16, the monitoring system 500 includes the emotion estimation apparatus 200 described above, an input unit 580, and an output unit 590. The monitoring system 500 is installed in a call center or the like, acquires incoming calls from the customer to the call center and outgoing calls from the call center to the customer from the call database 2, and detects a complaint call from a call record or the like.

  The input unit 580 is a data transfer interface or the like. For example, by connecting the call database 2 that stores all call center call records and the like to the input unit 480, the monitoring system 500 can receive the stored call records as target audio data. When stored in the call database 2, the target voice data is added with a telephone identifier, an operator identifier, a start time of the target voice data, an end time of the target voice data, and the like corresponding to the target voice data. Shall be.

  The emotion estimation device 200 estimates the emotion of the target voice data for one call, and outputs the telephone identifier, the operator identifier, the start time of the target voice data, the end time of the target voice data, the estimation result, etc. Output to 590.

  The output unit 590 is, for example, a display, a printer, or the like, and data received on a screen or paper (telephone identifier, operator identifier, start time of target voice data, end time of target voice data, estimation result thereof, etc.) Is displayed (see FIG. 17).

<Effect>
Searching for a claim call while listening to a large number of call records requires a very large human cost, but with such a configuration, a claim call can be automatically detected, and a call database is created based on the output result. Can easily search for a claim call. Analyzing the complaint calls searched in this way leads to the discovery of strong customer demands and dissatisfactions, malfunctions and problems with products and services. It also leads to the discovery of problems with operator interaction that cause complaint calls.

[Other variations]
As the emotion estimation device, a device provided with a voice recognition unit in front of the emotion estimation device 300 of the third embodiment or the emotion estimation device of the first embodiment may be used.

<Program and recording medium>
The emotion estimation apparatus and monitoring system described above can also be functioned by a computer. In this case, the program for causing the computer to function as the target device (the device having the functional configuration shown in the drawings in each embodiment) or each process of the processing procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

90 learning text corpus 91 learning speech corpus 100, 200, 300 emotion estimation device 210 speech recognition unit 320 prosodic feature quantity calculation unit 130 text analysis unit 131 morpheme analysis unit 133 syntax analysis unit 140, 240 explicitness estimation unit 141 241 abbreviation estimation unit 143, 243 replacement word estimation unit 144, 244 explicit word detection unit 144a explicit word list storage unit 145, 245 explicit feature quantity calculation unit 150, 350 classifier generation unit 160, 360 emotion identification unit 400, 500 Monitoring system 480, 580 Input unit 490, 590 Output unit

Claims (10)

Using at least one of the morphological analysis result and the syntax analysis result of the target text data, and an explicitness estimation unit that estimates an explicit feature amount indicating how much the semantic content is clearly indicated in the target text data;
Whether anger emotion is expressed in the target text data using a discriminator learned in advance from the explicit feature quantity of the text data for learning with a teacher signal indicating whether anger emotion is expressed An emotion identifying unit that identifies whether or not from the explicit feature quantity of the target text data estimated by the explicit estimation unit ,
The explicit feature amount is:
(1) Number of omissions for each abbreviation
(2) Presence / absence of each abbreviation
(3) The total number of omissions for all abbreviations included in the emotion estimation unit, not for each abbreviation
(4) Presence or absence of abbreviations in emotion estimation units
(5) Number of occurrences for each replacement word
(6) Presence / absence of each replacement word
(7) The total number of occurrences of all replacement words included in the emotion estimation unit, not for each replacement word
(8) Presence or absence of replacement word in emotion estimation unit
(9) Number of appearances for each explicit word
(10) Presence / absence of each explicit word
(11) The total number of appearances of all explicit words included in the emotion estimation unit, not for each explicit word
(12) Presence of explicit words in emotion estimation units
Or any combination thereof.
Emotion estimation device. The emotion estimation device according to claim 1,
The explicitness estimation unit includes:
Using the parsing result, and having an abbreviated case estimation unit for estimating a case that should be omitted in the target text data.
Emotion estimation device. The emotion estimation apparatus according to claim 1 or 2, wherein
The explicitness estimation unit includes:
Using the morphological analysis result, and having a replacement word estimation unit that estimates a word that exists in place of the word that should originally exist in the target text data,
Emotion estimation device. The emotion estimation apparatus according to any one of claims 1 to 3,
The explicitness estimation unit includes:
In a normal utterance, an explicit word list storage unit that stores in advance words that are easily omitted;
Using the morpheme analysis result, referring to the explicit word list storage unit, and having an explicit word detection unit for detecting an explicit word existing in the target text data,
Emotion estimation device. The emotion estimation apparatus according to any one of claims 2 to 4,
The explicitness estimation unit includes:
An explicit feature quantity calculation unit for calculating the explicit feature quantity using at least one of an estimation result of the abbreviation case estimation unit, an estimation result of the replacement word estimation unit, and a detection result of the explicit word detection unit; In addition,
Emotion estimation device. The emotion estimation apparatus according to any one of claims 1 to 5,
Speech that performs speech recognition processing on the target speech data and obtains the target text data obtained from the target speech data and the recognition reliability indicating the reliability of the recognition result for each word included in the target text data A recognition unit,
The explicitness estimation unit uses at least one of the morphological analysis result and the syntax analysis result of the target text data with the recognition reliability, and how much the semantic content is clearly specified in the target text data. Estimating explicit features that represent
Emotion estimation device. The emotion estimation apparatus according to claim 6,
Using the target speech data, further comprising a prosodic feature quantity calculating unit for calculating the prosodic feature quantity;
In addition to the explicit feature amount of the learning text data, the emotion discriminating unit uses a discriminator previously learned by using the prosodic feature amount of the learning speech data corresponding to the learning text data, Whether anger emotion is expressed in the target speech data is identified from the explicit feature amount estimated by the explicitness estimation unit and the prosodic feature amount calculated by the prosodic feature amount calculation unit To
Emotion estimation device. The explicitness estimation unit estimates an explicit feature amount indicating how much the semantic content is clearly indicated in the target text data by using at least one of the morphological analysis result and the syntax analysis result of the target text data. An explicitness estimation step;
An anger emotion is included in the target text data using a discriminator previously learned from the explicit feature quantity of the text data for learning with a teacher signal indicating whether the anger emotion is expressed by the emotion identification unit An emotion identification step that identifies whether or not is expressed from the explicit feature quantity of the target text data estimated in the explicitness estimation step,
The explicit feature amount is:
(1) Number of omissions for each abbreviation (2) Presence / absence of omission for each abbreviation (3) Total number of omissions of all abbreviations included in the emotion estimation unit instead of each abbreviation (4) In the emotion estimation unit Presence / absence of abbreviation (5) Number of occurrences for each replacement word (6) Presence / absence for each replacement word (7) Total number of occurrences of all replacement words included in the emotion estimation unit instead of each replacement word (8) Presence / absence of replacement word in emotion estimation unit (9) Number of appearances for each explicit word (10) Presence / absence for each explicit word (11) Not all explicit words but all explicit words included in emotion estimation unit Total number of appearances (12) Any of presence / absence of explicit word in emotion estimation unit, or a combination thereof,
Emotion estimation method. Program for causing a computer to function as the emotion estimation apparatus according to claims 1 to claim 7.   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images