JP2010224153A - Speech interaction device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To create appropriate response to user's utterance, even when speech recognition results or feeling estimation results is incorrect. <P>SOLUTION: The speech interaction device includes: a speech recognition unit 10 which recognizes speech uttered by the user, to extract a word included in the speech, and which calculates reliability of the word included in the speech; a feeling estimation unit 20 for estimating reliability of the estimated feeling by estimating the feeling of user's speech by using recognition results or rhythm information of the speech; a response candidate creation unit 30 for creating a response candidate by using the extracted word and a predetermined response template, and creating the response candidate by using the estimated feeling and the predetermined response template; and a response candidate selection unit 40 for selecting a response candidate based on a word or feeling, whose reliability is the highest, in the response candidates created by the response candidate creation unit 30. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a voice interaction apparatus and a program.

  2. Description of the Related Art Conventionally, there has been proposed a dialogue processing apparatus that performs conversations rich in variations depending on the emotional state of a user (see, for example, Patent Document 1). The dialogue processing device of Patent Literature 1 estimates a user's emotion using the concept of words / phrases input from the user, prosodic information, user's face image, and user's physiological information, and the user is based on emotion information representing the emotion. Generate an output statement to output to.

JP 2001-215993 A

  The dialogue processing apparatus of Patent Literature 1 estimates a user's emotion using a lot of information such as a phrase concept and prosodic information. However, it is very difficult to reliably estimate the user's emotion, and a wrong emotion may be estimated. However, since the dialogue processing apparatus of Patent Document 1 does not know the reliability of the emotion even if the wrong emotion is estimated, there is a problem that an output sentence is generated based on the wrong emotion estimation result.

  In addition, there is a known technique for generating a response based on the recognition result for the speech uttered by the user. However, when there is a misrecognition, there is a problem that an incorrect response is generated for the user utterance.

  The present invention has been proposed in order to solve the above-described problem, and provides a voice interaction apparatus and a program for generating an appropriate response to a user's utterance even if a voice recognition result or an emotion estimation result is incorrect. The purpose is to do.

  The voice interaction device according to the present invention recognizes a voice spoken by a user, extracts a word included in the voice, and calculates a reliability of the word included in the voice; and the voice recognition means A speech estimation unit that estimates the emotion of the user's speech using the recognition result of the speech or the prosodic information of the speech, calculates the reliability of the estimated emotion, and a word extracted by the speech recognition unit A response candidate is generated using a first response candidate generation unit that generates a response candidate using the response template, an emotion estimated by the emotion estimation unit, and a predetermined response template. Response candidate selection that selects a response candidate based on a word or emotion having the highest reliability among the response candidates generated by two response candidate generation means and the first and second response candidate generation means It includes a stage, a.

  According to the said invention, a response candidate is produced | generated using the word extracted by the speech recognition means, and the predetermined response template, and using the emotion estimated by the emotion estimation means, and the predetermined response template A response candidate is generated, and a response candidate based on the word or emotion having the highest reliability is selected from the generated response candidates. As a result, even if there is an error in the speech recognition result or the emotion estimation result, the above invention selects a response candidate that is not affected by the error, so that it is possible to generate an appropriate response to the user's utterance. .

  Even if there is an error in the speech recognition result or the emotion estimation result, the present invention generates a response that is not affected by the error, so that an appropriate response can be made to the user's utterance.

It is a block diagram which shows the structure of the voice interactive apparatus which concerns on embodiment of this invention. It is a figure which shows a response production | generation rule. It is a flowchart which shows the voice dialogue routine performed by the voice dialogue apparatus.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a voice interaction apparatus according to an embodiment of the present invention. The voice interactive apparatus estimates a user's emotion based on a voice recognition unit 10 that recognizes voice, a voice recognition history storage unit 11 that stores a history recognized by the voice recognition unit 10, and a recognition result of the voice recognition unit 10. And an emotion estimation history storage unit 21 for storing a history of emotion estimation results of the emotion estimation unit 20.

  Furthermore, the voice interactive apparatus stores a response candidate generation unit 30 that generates response candidates, a response rule storage unit 31 that stores response rules, a response candidate selection unit 40 that selects response candidates, and a response history. A response history storage unit 41.

  The speech recognition unit 10 performs recognition processing of speech uttered by the user, recognizes one or more words included in the speech, and calculates the reliability of each word. The words recognized by the voice recognition unit 10 and their reliability are sequentially stored in the voice recognition history storage unit 11.

  The reliability calculation method is not particularly limited. For example, Document 1 “Reliability calculation method based on fast word posterior probabilities in the two-pass search algorithm” Li et al., December 19, 2003, The technology described in the Information Processing Society of Japan Research Report can be used. In the present embodiment, the reliability is 0 to 1.0, and the highest reliability is 1.0.

  When the voice recognition unit 10 recognizes the user's voice, the emotion estimation unit 20 estimates the emotion of the input voice based on the voice recognition result output from the voice recognition unit 10. The emotion estimation method using the speech recognition result is not particularly limited. For example, Reference 2 “Emotion estimation based on the emotion-causing factor corpus acquired from the Web”, Tokuhisa et al. Proceedings of the next conference, pp. 33-36, March 2008, can be used.

  In addition, when the voice recognition unit 10 does not recognize the user's voice, the emotion estimation unit 20 estimates the user's emotion using prosodic information (basic frequency, etc.) included in the user's voice. The technique for estimating emotions using prosodic information is not particularly limited, but for example, the technique described in Japanese Patent Application Laid-Open No. 2002-91482 (emotion detection method, emotion detection apparatus, and storage medium) may be used. it can.

  When the voice recognition unit 10 recognizes a voice, the emotion estimation unit 20 may not only estimate the emotion using the voice recognition result but also may estimate the emotion using prosodic information.

  When the speech recognition unit 10 obtains a speech recognition result with high reliability, the response candidate generation unit 30 generates a response candidate based on the recognized word. Moreover, the response candidate production | generation part 30 produces | generates the response candidate based on the estimated emotion, when an emotion is estimated in the emotion estimation part 20. FIG.

  Note that the response candidate generation unit 30 generates a response candidate even when a voice recognition result is not obtained and an emotion is not estimated or when no voice is input for a predetermined time (in the case of lime out). These response candidates are generated according to the response generation rules stored in the response generation rule storage unit 31, respectively.

  FIG. 2 is a diagram illustrating response generation rules stored in the response generation rule storage unit 31. The response generation rule associates an input with a response template for generating a response candidate using the input.

  [Verb], [adjective], and [noun] illustrated in FIG. 2 respectively indicate a verb, an adjective, and a noun recognized by the speech recognition unit 10. [Emotion: Fun] and [Emotion: Sad] indicate the user's emotions estimated by the emotion estimation unit 20, respectively. “No voice recognition candidate, no emotion estimation result” means that the voice recognition unit 10 cannot obtain a voice recognition result with high reliability (for example, a voice recognition result including a word whose reliability exceeds a threshold), and the emotion estimation unit 20 It shows that emotion is not estimated. [Timeout] indicates a case where the user does not speak for a predetermined time after the sound is output from the apparatus.

  Here, according to FIG. 2, the response generation rule includes a verb and three response templates (“[verb]” and “[verb]” for generating response candidates using the verb. ? "And" Where did [verb]? ") Associated.

  When the verb “eat” is recognized by the voice recognition unit 10, [eat] is transformed into an optimal format and inserted into the [verb] portion of the response template. As a result, three response candidates are generated: “I ate it.”, “Who ate with?” And “Where did you eat?”.

  The response generation rule associates an emotion, for example, “fun” with two response templates for generating response candidates using the emotion (“It was good”, “It was fun”). Yes. When the emotion estimation unit 20 estimates the emotion “fun”, two response candidates “good” and “it was fun” are generated.

  Response generation rule is, the character string "Hello", which is recognized by the voice recognition unit 10, in association with the "Hello". Therefore, if the string "Hello" is recognized by the voice recognition unit 10, "Hello" is generated as the response candidate.

  The response generation rule associates the character string “What is today's weather?” Recognized by the voice recognition unit 10 with “Today's weather is [Today's weather information]”. Therefore, when the character string “What is today's weather?” Is recognized by the voice recognition unit 10, today's weather information (for example, “cloudy”) is acquired from the outside, and “Today's weather is cloudy” as a response candidate. Is generated.

  The response generation rule associates [speech recognition candidate, no emotion estimation result] with “Please say again.”, And associates [timeout] with “the weather is good today”. Therefore, when the speech recognition unit 10 cannot obtain a highly reliable speech recognition result and the emotion estimation unit 20 does not estimate the emotion, “Please say again” is generated as a response candidate. , “Today's weather is good” is generated as a response candidate.

  And the response candidate production | generation part 30 calculates the reliability of each response candidate, when the response candidate based on a speech recognition result and the response candidate based on an emotion are produced | generated. The reliability of the response candidate is the same value as the reliability of the word included in the response candidate or the reliability of the emotion of the response candidate.

  The response candidate selection unit 40 refers to the response history of the response history storage unit 41, excludes response candidates selected in the past from one or more response candidates generated by the response candidate generation unit 30, The response candidate with the highest reliability is selected from the remaining response candidates.

The voice interactive apparatus configured as described above is, for example, that the user “has gone to an amusement park today”.
The following voice dialogue routine is executed.

  FIG. 3 is a flowchart showing a voice dialogue routine executed by the voice dialogue apparatus.

  In step S1, the voice recognition unit 10 determines whether or not there is a voice input from the user until a predetermined time elapses after the execution of this routine is started or after the voice reproduction of the apparatus is performed. If the determination is affirmative, the process proceeds to step S3. If the determination is negative, it is determined that a timeout has occurred, and the process proceeds to step S2.

  In step S2, the response candidate generator 30 generates a response that prompts the user to provide information, for example. Specifically, the response candidate generation unit 30 generates “Today is good weather” associated with [timeout] as a response candidate in accordance with the response generation rule stored in the response generation rule storage unit 31. To do. Then, the process proceeds to step S15.

  In step S <b> 3, the voice recognition unit 10 performs voice recognition processing on the voice spoken by the user “I went to an amusement park today”, extracts words from the input voice, and sets the reliability of each word. calculate.

  For example, in the present embodiment, the speech recognition unit 10 obtains the following recognition candidates 1 to 3 as a result of the speech recognition process.

Recognition candidate 1: “I went swimming today.”
Recognition candidate 2: “I went to an amusement park today.”
Recognition candidate 3: “I was swimming today.”

And the speech recognition part 10 outputs the recognition candidate 1 which becomes the maximum likelihood among the recognition candidates 1-3 as a speech recognition result. The reliability of each word of the speech recognition result is calculated based on the technique of Document 1 described above, for example. As a result, in the present embodiment, for example, the following speech recognition result is obtained.
“Today (0.7) was (0.6) swimming (0.2) and (0.4) going to (0.8) (0.8) to (1.0).”

  The number in parentheses indicates the reliability of the word immediately before (left side).

  In step S4, the speech recognition unit 10 determines whether there is an independent word whose reliability is higher than a threshold value (for example, 0.5) among the recognized words. If the determination is affirmative, the process proceeds to step S6. If the determination is negative, the process proceeds to step S5.

  In addition, when the recognition result shown in step S3 is obtained, there are “today” and “done” as words having a reliability higher than 0.5 in the recognition result. These two words are both independent words. Therefore, in the present embodiment, since the recognition result includes an independent word whose reliability is higher than the threshold, the process proceeds to step S6.

  In step S5, since the emotion estimation unit 20 cannot use the speech recognition result of the speech recognition unit 10, the emotion estimation unit 20 estimates the user's emotion using the prosodic information of the speech input to the speech recognition unit 10. Here, for example, a technique described in JP-A-2002-91482 is used. In the present embodiment, for example, “fun” is estimated as the emotion, and the reliability is assumed to be 1.0. Then, the process proceeds to step S6.

  In step S <b> 6, the response candidate generation unit 30 generates a response candidate using the estimation result of the emotion estimation unit 20 according to the response generation rule stored in the response generation rule storage unit 31, and sets the reliability of each response candidate. calculate.

  In the case of the present embodiment, response candidates associated with [emotion: fun] of the response generation rule, “Good”, “It was fun” are generated, and each reliability is 1.0. is there.

  In step S <b> 7, the response candidate generation unit 30 generates a response candidate using the speech recognition history stored in the speech recognition history storage unit 11. Here, the response candidate generation unit 30 searches the independent words (words) having high reliability (for example, reliability of 0.5 or more) from the voice recognition history retroactively from the current time. And the response candidate production | generation part 30 produces | generates a response candidate based on the searched word and the response template matched with the word.

  On the other hand, in step S8, the voice recognition unit 10 stores the voice recognition result shown in step S3 in the voice recognition history storage unit 11. Then, the process proceeds to step S9.

  In step S9, the emotion estimation unit 20 estimates the emotion of each recognition candidate by using the technique described in Document 2 for each recognition candidate used in the speech recognition process in step S3. And the emotion estimation part 20 estimates the emotion which occupied many in the emotion of each recognition candidate as a user's emotion, and calculates the ratio for which the emotion accounts as reliability.

  In the present embodiment, the emotions of the recognition candidates 1 to 3 shown in step S3 are all estimated to be “fun”, for example. In this case, since “fun” accounts for three of the three candidates, the reliability of “fun” is 3/3 = 1.0. Then, the process proceeds to step S10.

  If two emotions among the recognition candidates 1 to 3 are “fun” and the remaining emotion is estimated to be “sad”, “fun” is estimated as the user's emotion, and its reliability Is 2/3 = 0.67.

  In step S10, the response candidate generation unit 30 generates a response candidate using the emotion estimation result of the emotion estimation unit 20 according to the response generation rule stored in the response generation rule storage unit 31, and the reliability of each response candidate. Is calculated.

  In the case of the present embodiment, response candidates associated with [emotion: fun] of the response generation rule, “Good”, “It was fun” are generated. Both of these reliability levels are 1.0.

  In step S11, the response candidate generation unit 30 generates a response candidate using the speech recognition result of the speech recognition unit 10, and calculates the reliability of each response candidate.

  For example, in the present embodiment, the response candidate generation unit 30 generates a response candidate using each word of the speech recognition result shown in step S3.

  According to the response generation rule shown in FIG. 2, for “Today”, for example, response candidates associated with [Noun], “Today?”, “What is today?”, “Who is today?” Are generated. The Further, for example, for “performed”, response candidates associated with [verb], “has gone”, “who did you go”, and “where did you go” are generated.

  Further, according to step S3, since the reliability of “Today” is 0.7, each of the reliability candidates “Today?”, “What is today?”, “Who is today?” Is 0. .7. Similarly, since the reliability of “I went” is 0.8, the reliability of each of the response candidates “I went”, “Who did you go?”, And “Where did you go?” 0.8. In addition, the response candidate production | generation part 30 produces | generates a response candidate similarly about the other word contained in a speech recognition result. Then, the process proceeds to step S12.

  In step S <b> 12, the response candidate selection unit 40 performs a response candidate filtering process on the response candidates generated by the response candidate generation unit 30 using the response history stored in the response history storage unit 41. Specifically, the response candidate selection unit 40 excludes response candidates that have been selected in the past as a response history from the response candidates generated by the response candidate generation unit 30. Thereby, it is possible to avoid the same response as in the past. Then, the process proceeds to step S13.

  In step S13, the response candidate selection unit 40 determines whether there is a response candidate, that is, whether a response candidate remains even after the above-described step S12. If the determination is affirmative, the process proceeds to step S15. If the determination is negative, the process proceeds to step S14.

  In step S <b> 14, the response candidate generation unit 30 generates a response candidate that prompts reconciliation or re-input. Specifically, the response candidate generation unit 30 generates “Please say again” associated with “response recognition candidate, no emotion estimation result” of the response generation rule as a response candidate. Note that the response candidate generation unit 30 may generate the answers “Yes”, “Sodane”, and the like instead of this response candidate. Then, the process proceeds to step S17.

  On the other hand, in step S15, the response candidate selection unit 40 selects a response candidate having the highest reliability from the already generated response candidates. When there are a plurality of response candidates with the highest reliability, the response candidate selection unit 40 may select response candidates according to a predetermined priority, or may select response candidates at random. .

  In the present embodiment, since the reliability (= 1.0) of the response candidates generated in step S10, “It was good” and “It was fun” is the highest, the response candidate selection unit 40 is “good. "Ne." Or "It was fun." Then, the process proceeds to step S16.

  In step S16, the response candidate selection unit 40 stores the response candidate selected in step S15 in the response history storage unit 41 as a response history. Then, the process proceeds to step S17.

  In step S15, the response candidate selection unit 40 performs speech synthesis on the response candidate obtained in any of steps S2, S14, and S16, and reproduces the speech. And it returns to step S1 and will be in the state of waiting for a user's speech.

  As described above, the voice interaction apparatus according to the embodiment of the present invention generates response candidates based on the speech recognition result, generates response candidates based on the emotion estimation result, The response candidate with the highest reliability is output from the response candidates. As a result, even if there is an error in the speech recognition result or the emotion estimation result and the response candidate is generated, the above-described voice interaction device outputs the most reliable response candidate without the influence, thereby preventing an erroneous response. You can respond and have a smooth conversation with the user.

  It should be noted that the present invention is not limited to the above-described embodiment, and it is needless to say that the present invention can also be applied to a design modified within the scope described in the claims.

  For example, even when the speech recognition unit 10 obtains a speech recognition result, the emotion estimation unit 20 may estimate the user's emotion using prosodic information of the user's speech. In this case, when the user's emotion based on the speech recognition result matches the emotion based on the prosodic information, the emotion estimation unit 20 may output the matched emotion as an estimation result.

  The voice interaction apparatus shown in FIG. 1 may be configured by installing a program for executing the voice interaction routine shown in FIG. 3 in a computer.

10 speech recognition unit 11 speech recognition history storage unit 20 emotion estimation unit 30 response candidate generation unit 31 response generation rule storage unit 40 response candidate selection unit 41 response history storage unit

Claims (4)

Voice recognition means for recognizing a voice spoken by a user, extracting a word included in the voice, and calculating a reliability of the word included in the voice;
Emotion estimation means for estimating the emotion of the user's voice using the recognition result of the voice recognition means or the prosodic information of the voice, and calculating the reliability of the estimated emotion;
First response candidate generation means for generating a response candidate using a word extracted by the voice recognition means and a predetermined response template;
Second response candidate generation means for generating a response candidate using the emotion estimated by the emotion estimation means and a predetermined response template;
Among the response candidates generated by the first and second response candidate generation means, a response candidate selection means for selecting a response candidate based on the word or emotion having the highest reliability,
Spoken dialogue device with The voice interaction apparatus according to claim 1, wherein the voice recognition unit extracts only words whose reliability of each word included in the voice is a predetermined threshold value or more. Response history storage means for storing the response candidate selected by the response candidate selection means as a response history;
The response candidate selection unit excludes the response candidates included in the response history from the response candidates generated by the first and second response candidate generation units, and has the highest reliability from the remaining response candidates The voice interaction device according to claim 1 or 2, wherein a response candidate based on a word or emotion is selected.   A voice dialogue program for causing a computer to function as each means of the voice dialogue apparatus according to any one of claims 1 to 3.

Images