JP2008286930A - Voice interactive device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To smoothly progress interaction with a user by appropriately setting system utterance. <P>SOLUTION: Prediction recognition processing for recognizing user's utterance is performed based on a prediction sentence (Step 2), and large vocabulary recognition processing for recognizing user's utterance is performed based on a large vocabulary dictionary (Step 3). A prediction likelihood which is calculated in the prediction recognition processing is compared with a large vocabulary likelihood which is calculated in the large vocabulary recognition processing (Step 4), after the prediction likelihood is determined to be more than the large vocabulary likelihood, a partial word recognition processing for recognizing the user's utterance is performed (Step 6). Then, a prediction recognition result of a sentence unit, which is calculated in the prediction recognition processing, is compared with a partial word recognition result of a word unit, which is calculated in the partial word recognition result, and a matching degree of the recognition result (complete matching, partial matching and complete mismatching) is determined (Step 7). According to the determined matching degree, a content of the system utterance is separately set (Step 8 to 10). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a voice interaction apparatus that recognizes and responds to a user utterance.

In recent years, in a field such as a car navigation system and a call center, a voice recognition device has been developed that automatically recognizes a voice uttered by a user (hereinafter referred to as a user utterance) and executes various processes. In such a speech recognition apparatus, after extracting a feature amount of the input speech by speech analysis, the contents of the user utterance are recognized by searching for a sequence of acoustic models matching the feature amount of the input speech. I am doing so. In addition, in order to improve the recognition accuracy of user utterances, a language model that sets word occurrence probabilities based on language statistics, grammar, etc. is set, and an acoustic model that matches the input speech under the constraints of this language model Search for a column. In addition to simply recognizing the user utterance and outputting the recognition result, by setting the voice uttered from the device side according to the content of the user utterance (hereinafter referred to as system utterance), the user and the device A voice dialogue apparatus has also been developed that allows dialogue to proceed between the two (for example, see Patent Document 1).
JP 2001-175279 A

  However, in the speech dialogue apparatus described in Patent Document 1, the content of the system utterance responding to the user utterance is divided into a case where the recognition of the user utterance is successful and a case where the recognition of the user utterance fails. I am trying to set it. As described above, if the content of the system utterance is simply set based on whether or not the user utterance has been successfully recognized, it becomes difficult to appropriately set the content of the system utterance. It may be difficult to proceed and the user may feel annoyed. In other words, considering the case where the user utterance recognition fails, there are cases where the user utterance fails without being able to recognize a part of the user utterance or when the user utterance fails without being able to recognize all of the user utterances. Setting system utterances with the same content has been a factor that hinders smooth dialogue with users.

  An object of the present invention is to smoothly advance a dialog with a user by appropriately setting a system utterance.

  The speech dialogue apparatus of the present invention includes a prediction recognition unit that recognizes a user utterance based on the predicted utterance content and outputs a prediction recognition result, and a word that recognizes the user utterance based on a predetermined word dictionary and outputs a word recognition result Recognizing means, coincidence degree determining means for comparing the prediction recognition result with the word recognition result to determine the coincidence degree of the recognition result, and system utterance for setting the content of the system utterance based on the coincidence degree of the recognition result And setting means.

  The spoken dialogue apparatus according to the present invention is characterized in that the coincidence degree determining means determines the coincidence degree as one of complete coincidence, partial coincidence or complete disagreement.

  The spoken dialogue apparatus of the present invention includes a large vocabulary recognition unit that recognizes a user utterance based on a predetermined large vocabulary dictionary and outputs a likelihood of a large vocabulary recognition result, and the likelihood of the large vocabulary recognition result and the prediction recognition. A likelihood determination means for comparing the likelihood of the prediction recognition result output from the means, and after determining that the likelihood of the prediction recognition result exceeds the likelihood of the large vocabulary recognition result, The degree of coincidence determination means starts the determination of the degree of coincidence.

  According to the present invention, the degree of coincidence between the prediction recognition result and the word recognition result is determined, and the content of the system utterance is set based on the determined degree of coincidence. It is possible to facilitate the dialogue with the user.

  FIG. 1 is a block diagram showing a configuration of a voice interaction apparatus 10 according to an embodiment of the present invention. As shown in FIG. 1, the voice interaction apparatus 10 includes a microphone 11 that converts a voice uttered by a user, that is, a user utterance, into an electric signal, and an acoustic analysis unit 12 that extracts a voice feature amount from the electric signal. Yes. In addition, the speech dialogue apparatus 10 is provided with a prediction recognition unit (prediction recognition unit) 13 that recognizes a user utterance, a large vocabulary recognition unit (large vocabulary recognition unit) 14, and a partial word recognition unit (word recognition unit) 15. A voice feature amount is input to each of the recognition units 13 to 15 via the acoustic analysis unit 12.

  Here, FIG. 2 is a block diagram showing the configuration of the prediction recognition unit 13, large vocabulary recognition unit 14, and partial word recognition unit 15. As shown in FIG. 2, each of the recognition units 13 to 15 includes acoustic models 16a to 16c that are statistical models (such as hidden Markov models) of speech feature patterns in units of phonemes and syllables, word connection probabilities, Language models 17a to 17c, which are statistical models of appearance probability (N-gram model or the like), are provided respectively. In addition, in order to define the vocabulary to be recognized and the pronunciation thereof, each of the recognition units 13 to 15 is provided with a dictionary 18a to 18c in which a large number of words are recorded. The words registered in the dictionaries 18a to 18c are collected from a large amount of text databases composed of newspaper articles, conference lectures, web pages, and the like.

In addition, each of the recognition units 13 to 15 uses the acoustic models 16a to 16c and the language models 17a to 17c to perform speech recognition processing (Journal of the Japanese Society for Artificial Intelligence Vol. Decoders 19a to 19c are provided for executing the work (see pages 41 to 49), and the input speech X is converted into an optimal word string W by the decoders 19a to 19c. This voice recognition process is a process for searching for a word string W that maximizes the posterior probability p (W | X) for the input voice X. Various word strings are obtained using the following equation (1) based on Bayes' theorem. This is a process of calculating the posterior probability p (W | X) for W and adopting the word string W with the highest posterior probability p (W | X) as the recognition result (see the following equation (2)). . This posterior probability p (W | X), that is, the likelihood is an index indicating how valid the recognition result of the input speech is. Note that p (X) shown in the denominator of equation (1) does not affect the determination of the word string W and can be ignored. Further, the probability p (W) in the expression (2) means language models 17a to 17c expressing the occurrence probability of the word string W, and the probability p (X | W) in the expression (2) is calculated from the word string W. The acoustic models 16a to 16c expressing the probability that the input speech X is obtained are meant.
p (W | X) = p (W) * p (X | W) / p (X) (1)
W = argmax p (W) * p (X | W) (2)

  Next, the prediction recognition process executed by the prediction recognition unit 13, the large vocabulary recognition process executed by the large vocabulary recognition unit 14, and the partial word recognition process executed by the partial word recognition unit 15 will be described. First, the prediction recognizing unit 13 predicts a user utterance uttered from the user based on the conversation scene, and registers the predicted user utterance in the dictionary 18a in sentence units. Next, the prediction recognition unit 13 divides the user utterances registered in the dictionary into words or phrase units, creates a language model 17a with the recognition target as a sentence unit, and then performs the speech recognition process described above under the restriction of the language model 17a. This is executed, and a sentence unit recognition result W1 (hereinafter referred to as a prediction recognition result) for the user utterance and a likelihood R1 (hereinafter referred to as a prediction likelihood) of the prediction recognition result W1 are output. Note that it is also possible to create a grammar by dividing user utterances registered in the dictionary into units of words or phrases, and execute speech recognition processing using this grammar. In this prediction recognition process, since recognition is performed after narrowing down the recognition target, it is possible to improve the recognition accuracy even though the recognition range is narrowed.

  Further, the large vocabulary recognition unit 14 creates a language model 17b in which a recognition target is a sentence unit using words in the dictionary (large vocabulary dictionary) 18b collected from the various text data described above. Then, the large vocabulary recognition unit 14 executes the speech recognition process described above under the constraint of the created language model 17b, and recognizes the sentence-by-sentence recognition result W2 (hereinafter referred to as the large vocabulary recognition result) for the user utterance, and the large vocabulary. The likelihood R2 of the recognition result W2 (hereinafter referred to as a large vocabulary likelihood) is output. In this large vocabulary recognition process, since speech recognition is performed without excessively narrowing down the recognition target, it is possible to widen the recognition range although the recognition accuracy is reduced.

  In addition, the partial word recognition unit 15 creates a language model 17c with the recognition target as a word unit using words in the dictionary (word dictionary) 18c collected from the various text data described above. Then, the partial word recognition unit 15 executes the speech recognition process described above under the restriction of the created language model 17c, and outputs a word-unit recognition result W3 (hereinafter referred to as a partial word recognition result) for the user utterance. That is, the above-described prediction recognition process and large vocabulary recognition process are processes for recognizing user utterances in sentence units, but this partial word recognition process is a process for recognizing user utterances in word units.

  Furthermore, the prediction recognition result W1 is input from the prediction recognition unit 13 to the partial word recognition unit 15 functioning as a degree of coincidence determination unit, and the partial word recognition unit 15 recognizes the prediction recognition result W1 and the word recognized in sentence units. The degree of coincidence with the partial word recognition result (word recognition result) W3 recognized in units is determined and output. That is, when the prediction recognition result W1 is “A, B, C” and the partial word recognition results W3 are “A”, “B”, “C”, the partial word recognition unit 15 determines that they are completely matched, When the prediction recognition result W1 is “A, B, C” and the partial word recognition results W3 are “A”, “B”, “D”, the partial word recognition unit 15 determines partial match, When the result W1 is “A, B, C” and the partial word recognition result W3 is “D”, “E”, “F”, the partial word recognition unit 15 determines that they are completely inconsistent. Note that A to F described above mean different words.

  Further, the speech dialogue apparatus 10 is provided with a prediction inside / outside determination unit 20 as a likelihood determination unit, and the prediction inside / outside determination unit 20 compares the prediction likelihood R1 with the large vocabulary likelihood R2 to determine whether or not It is determined whether or not the content of the utterance is within the prediction range of the prediction recognition unit 13. When the prediction likelihood R1 is larger than the large vocabulary likelihood R2, the determination result that the user utterance is within the prediction is output from the prediction inside / outside determination unit 20 to the system utterance selection unit 21 described later. On the other hand, when the prediction likelihood R1 is smaller than the large vocabulary likelihood R2, the determination result that the user utterance is not predicted is output from the prediction inside / outside determination unit 20 to the system utterance selection unit 21.

  Further, the speech dialogue apparatus 10 is provided with a system utterance selection unit 21 as a system utterance setting unit. The system utterance selection unit 21 includes a prediction recognition result W1, a large vocabulary recognition result W2, and a partial word recognition result W3. The degree of coincidence from the partial word recognition unit 15 and the determination result from the prediction inside / outside determination unit 20 are input. In this system utterance selection unit 21, a plurality of contents and grammars related to speech uttered from the apparatus side, that is, system utterances are registered, and the system utterance selection unit 21 selects the contents of the system utterance based on various kinds of input information. To do. Then, the system utterance selection unit 21 outputs the text data of the selected system utterance to the speech synthesizer 22, and the speech synthesizer 22 analyzes the text data and generates an electrical signal having a speech waveform. Next, an electrical signal is input from the speech synthesizer 22 to the speaker 23, and a system utterance is issued from the speaker 23 to the user.

  Subsequently, the execution procedure of the voice dialogue process by the voice dialogue apparatus 10 will be specifically described with reference to a flowchart. Here, FIG. 3 is a flowchart showing an execution procedure of the voice dialogue processing. FIG. 4A is an explanatory diagram showing an example of the prediction recognition result W1 and the large vocabulary recognition result W2 determined to be out of prediction, and FIG. 4B shows the prediction recognition result W1 determined to be within prediction and It is explanatory drawing which shows an example of the large vocabulary recognition result W2. Further, FIG. 5A is an explanatory diagram showing an example of the prediction recognition result W1 and the partial word recognition result W3 determined to be complete match, and FIG. 5B shows the prediction recognition result W1 determined to be partial match and FIG. 5C is an explanatory diagram illustrating an example of the partial word recognition result W3, and FIG. 5C is an explanatory diagram illustrating an example of the prediction recognition result W1 and the partial word recognition result W3 that are determined to be completely inconsistent.

  As shown in FIG. 3, in step S1, a user utterance (for example, “AB University”) is captured, in step S2, a prediction recognition process for the input voice is executed, and in a subsequent step S3, a large vocabulary recognition process for the input voice is executed. The Subsequently, it progresses to step S4 and it is determined whether the prediction likelihood R1 exceeds the large vocabulary likelihood R2. If it is determined in step S4 that the prediction likelihood R1 is less than the large vocabulary likelihood R2 (see FIG. 4A), it is determined that the content of the recognized user utterance is outside the prediction of the speech recognition apparatus. Therefore, the process proceeds to step S5, where a system utterance (for example, “Please say a genre”) for specifying the field of user utterance from a wide range is uttered, and the user utterance is captured again in step S1.

  On the other hand, when it is determined in step S4 that the prediction likelihood R1 exceeds the large vocabulary likelihood R2 (see FIG. 4B), the content of the recognized user utterance is within the prediction of the speech recognition apparatus. Therefore, the process proceeds to step S6 as it is, and the partial word recognition process for the input voice is executed. Next, the process proceeds to step S7, where the predicted recognition result W1 and the partial word recognition result W3 are compared, and the degree of coincidence (complete match, partial match, complete mismatch) of the recognition results W1, W3 is determined.

  In this step S7, when it is determined that the recognition results W1 and W3 are completely coincident (see FIG. 5A), the confirmation response of the recognition result is omitted and the system utterance of the next stage (for example, “Faculty” Where are you? ”). As described above, when the user utterance is determined to be within the prediction based on the prediction likelihood R1 and the large vocabulary likelihood R2, and it is determined that the prediction recognition result W1 and the partial word recognition result W3 are completely identical. Since it can be determined that the accuracy of the prediction recognition result W1 for the user utterance is extremely high, the conversation can be immediately advanced to the next stage so that the conversation can proceed without giving the user troublesome confirmation response or the like. Is possible.

  On the other hand, when it is determined in step S7 that the recognition results W1 and W3 are partially matched (see FIG. 5B), the user utterance is predicted based on the prediction likelihood R1 and the large vocabulary likelihood R2. This is a case where the prediction recognition result W1 and the partial word recognition result W3 are partially different. That is, since the accuracy of the prediction recognition result W1 with respect to the user utterance is in a slightly low state, the process proceeds to step S9 and partial matching is performed to determine the difference (undefined part) between the prediction recognition result W1 and the partial word recognition result W3. Processing is executed.

  Here, FIG. 6 is a flowchart showing an execution procedure of the partial matching process. As shown in FIG. 6, first, in step S21, a system utterance (for example, “What is AB?”) For prompting a user utterance regarding an uncertain part is uttered. In step S22, a user utterance (for example, “is a university”) is captured, and in step S23, an uncertain part is added to the dictionary, and a prediction recognition process for the input speech is executed. Next, in step S24, a system utterance (for example, “AB University”) for confirming the predicted knowledge result W1 to the user is uttered, and in a succeeding step S25, a user utterance (for example, “yes”) for an answer to the system utterance is issued. It is captured.

  Subsequently, in step S26, a prediction recognition process for the user utterance is executed, and in subsequent step S27, it is determined whether or not the recognition result of the user utterance is affirmative (eg, “yes”). If it is determined in step S27 that the recognition result is affirmative, it is determined that the difference of the partial word recognition result W3 with respect to the predicted recognition result W1 has been resolved, and the process proceeds to step S28, where the system utterance of the next stage (for example, "Where is your faculty?") On the other hand, if it is determined in step S27 that the recognition result is negative, it is determined that the difference between the predicted recognition result W1 and the partial word recognition result W3 has not been resolved, and the process proceeds to step S29, where the user again A system utterance prompting utterance (eg, “Please say again”) is uttered.

  As described above, when it is determined that the user utterance is within the prediction of the voice interaction apparatus 10 and the recognition results W1 and W3 are partially matched, the actual user utterance and the predicted recognition result W1. Are slightly different from each other, and therefore, a system utterance for determining this difference is set. Thereby, since a difference can be eliminated by a minimum dialogue, it is possible to make the dialogue proceed without causing discomfort to the user.

  As shown in FIG. 3, when it is determined in step S7 that the recognition results W1 and W3 are completely inconsistent (see FIG. 5C), the prediction likelihood R1 and the large vocabulary likelihood R2 This is a case where the user utterance is determined to be within the prediction based on the above, but the prediction recognition result W1 and the partial word recognition result W3 are completely different. That is, since the prediction recognition result W1 is out of the user's utterance, the process proceeds to step S10, and complete mismatch processing is performed to determine the difference between the prediction recognition result W1 and the partial word recognition result W3. The

  Here, FIG. 7 is a flowchart showing an execution procedure of complete mismatch processing. As shown in FIG. 7, first, in step S31, a system utterance (for example, “Are you a university? A bank?”, “AB? XY?”) For prompting a user utterance regarding an uncertain part is issued. . In step S32, user utterances (eg, “AB university”, “university”, “AB”) are captured, and in step S33, predictive recognition processing for the input speech is executed. Next, in step S34, a system utterance (for example, “AB university”, “university”, “AB” ”) for confirming the word recognized by the prediction recognition process to the user is uttered. In S35, a user utterance (for example, “Yes”) is captured.

  Subsequently, in step S36, a prediction recognition process for the user utterance is executed, and in subsequent step S37, it is determined whether or not the recognition result of the user utterance is affirmative (eg, “yes”). If it is determined in step S37 that the recognition result is affirmative, the difference between the partial word recognition result W3 and the predicted recognition result W1 has been eliminated, and the process proceeds to step S38, where the system utterance in the next stage is performed. (For example, “Where is your faculty?”) On the other hand, if it is determined in step S37 that the recognition result is negative, the difference between the predicted recognition result W1 and the partial word recognition result W3 has not been resolved. A system utterance prompting the user to speak again (for example, “Please say again”) is uttered.

  As described above, when it is determined that the user utterance is within the prediction of the voice interaction apparatus 10 and the recognition results W1 and W3 are completely inconsistent, the actual user utterance and the predicted recognition result W1. Are different from each other in the case of partial matching, and therefore, a system utterance is set to determine this difference. Thereby, since a difference can be eliminated by a minimum dialogue, it is possible to make the dialogue proceed without causing discomfort to the user.

  As described so far, by comparing the prediction likelihood R1 and the large vocabulary likelihood R2, it is determined whether or not the content of the user utterance is within the prediction of the voice interaction apparatus 10, and according to this determination result Thus, since the contents of the system utterance are changed, the recognition process for the user utterance can be appropriately executed even when the user utterance is not narrowed down.

  Further, the content of the subsequent system utterance is changed in accordance with the degree of coincidence between the prediction recognition result W1 in sentence units obtained by the prediction recognition process and the partial word recognition result W3 in word units obtained by partial word recognition. Therefore, an appropriate system utterance can be set according to the degree of recognition of the user utterance, and the conversation with the user can proceed smoothly. That is, even if the recognition results W1 and W3 are different, whether the recognition results W1 and W3 are partially different or whether the recognition results W1 and W3 are completely different. Accordingly, it is possible to change the contents of the system utterance that is uttered in order to lead the recognition result to the correct answer, so that the difference between the recognition results W1 and W3 can be eliminated by a minimum dialogue. Thereby, it becomes possible to advance a conversation smoothly, without giving a user discomfort.

  It goes without saying that the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention. The illustrated voice interaction device 10 can be effectively applied in various interactive fields such as a car navigation system and a call center.

It is a block diagram which shows the structure of the voice interactive apparatus which is one embodiment of this invention. It is a block diagram which shows each structure of a prediction recognition part, a large vocabulary recognition part, and a partial word recognition part. It is a flowchart which shows the execution procedure of a voice interaction process. (A) is explanatory drawing which shows an example of the prediction recognition result and large vocabulary recognition result determined to be outside prediction, (B) is an explanation which shows an example of the prediction recognition result and large vocabulary recognition result determined to be in prediction FIG. (A) is explanatory drawing which shows an example of the prediction recognition result and partial word recognition result which are determined to be exact match, (B) is an explanation which shows an example of the prediction recognition result and partial word recognition result which are determined to be partial match (C) is an explanatory view showing an example of a prediction recognition result and a partial word recognition result determined to be a complete mismatch. It is a flowchart which shows the execution procedure of a partial matching process. It is a flowchart which shows the execution procedure of complete mismatch processing.

Explanation of symbols

10 Spoken Dialogue Device 13 Prediction Recognition Unit (Prediction Recognition Means)
14 Large vocabulary recognition section (large vocabulary recognition means)
15 Partial word recognition unit (word recognition means, coincidence determination means)
18a Dictionary (Large Vocabulary Dictionary)
18b Dictionary (word dictionary)
20 Predicted inside / outside determination unit (likelihood determination means)
21 System utterance selection part (system utterance setting means)

Claims (3)

A prediction recognition means for recognizing a user utterance based on the predicted utterance content and outputting a prediction recognition result;
Word recognition means for recognizing a user utterance based on a predetermined word dictionary and outputting a word recognition result;
A degree of coincidence determination means for comparing the prediction recognition result and the word recognition result to determine the degree of coincidence of the recognition result;
And a system utterance setting unit configured to set the contents of the system utterance based on the degree of coincidence of the recognition results. The voice interaction apparatus according to claim 1, wherein
The coincidence degree determining means determines the degree of coincidence according to any of perfect coincidence, partial coincidence, and complete disagreement. The voice interaction apparatus according to claim 1 or 2,
Large vocabulary recognition means for recognizing a user utterance based on a predetermined large vocabulary dictionary and outputting the likelihood of a large vocabulary recognition result;
Likelihood determining means for comparing the likelihood of the large vocabulary recognition result with the likelihood of the prediction recognition result output from the prediction recognition means;
The speech dialogue apparatus according to claim 1, wherein after the likelihood of the prediction recognition result is determined to be greater than the likelihood of the large vocabulary recognition result, the coincidence degree determination unit starts the determination of the coincidence degree.

Images