JP2003228393A - Device and method for voice interaction, voice interaction program and recording medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make interaction with a user smoothly performable by precisely grasping the contents of a demand made by the user without limiting user's utterance. <P>SOLUTION: The device is provided with a voice recognizing means 203 which obtains two recognition results by inputting a user's voice signal and recognizing a voice by two different grammars for interaction, a language understanding means 140 which analyzes the two recognition results respectively and generates two kinds of understanding states consisting of item names, values, and likelihoods of the values corresponding to the two recognition results, and a recognition grammar selecting means 150 which determines which of the two kinds of understanding states is employed by comparing the reliabilities of the two recognition results. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice interactive apparatus and method for exchanging a voice interaction with a person to perform a predetermined operation such as schedule management, weather information guidance and car navigation, a program therefor, and a program therefor. The present invention relates to a recording medium recording the program.

[0002]

2. Description of the Related Art Hitherto, there has been known a so-called voice dialog device which achieves a predetermined operation by a dialog with a user. Such a voice interaction device has a voice recognition unit that converts a voice generated by a person into a character string, and controls the utterance content of the next device by using the result and the history of the conversation so far,
The utterance content is output as voice by the voice output unit.

Normally, the voice interactive apparatus repeats the process of deciding the request content for the user's device through conversation and performing the operation according to the confirmed user request content. There are two types of exchange methods that are performed to determine the content requested by the user. One is called "system driven dialogue" and the other is called "user driven dialogue". In system-driven dialogue, a strong limit is placed on the range in which the user can speak in response to confirmations and questions from the device side. For example, when the device side confirms or asks the prefecture name, only the prefecture name is accepted as the utterance of the user. on the other hand,
In the user-driven dialogue, the user's answer does not have such a restriction, and free utterance is allowed.

In the system-driven dialogue, since the degree of freedom of the user's utterance is low, the grammar for recognition may include a limited range of vocabulary. On the other hand, in user-driven dialogue, it is necessary to allow free speech, so
We must prepare a grammar for recognition that includes a wide vocabulary.

The recognition by the system-led grammar has a strong limitation, so that more accurate recognition can be expected, but there is a problem that the user cannot speak freely. On the other hand, in the recognition by the user-led grammar, although the user's free utterance is recognized, there is a problem that the recognition accuracy is lowered.

As a conventional technique for solving such a problem, there is a mixed initiative type. This is for switching each scene of the dialogue in order to take advantage of the high recognition accuracy in the system-led dialogue and the high degree of freedom of the user's speech in the user-led dialogue. For example, the following dialogue example 1 is an example of a dialogue of mixed initiative. <Dialogue example 1> Device utterance 1: “How do you meet your requirements?” ← Free question User utterance 2: “Tell us about the temperature” Device utterance 3: “When is the weather forecast?” ← Restricted question User Utterance 4: "Tomorrow"

However, in such a conventional technique, it has been necessary to judge the initiative of the dialogue and select the grammar to be used at the time when the device speaks, that is, before the user's utterance to be recognized is performed.

[0008]

The user does not always make the utterance according to the initiative intended by the device side. In particular, even if the device side assumes system initiative and prepares only a limited range of grammar, the user may try to transmit more information to the device. In this case, as a result, the user's intention is not accurately transmitted.

For example, consider the following dialogue example 2. <Dialogue example 2> Device Utterance 1: "What are your requirements?" User utterance 2: "Tell me about the temperature" Device Utterance 3: "When is the weather forecast?" User utterance 4: "Tell me about the temperature in Kanagawa"

As described above, even though the device intends that the user speaks the content related to the attribute "time" in (device utterance 3), in reality, as in (user utterance 4), In some cases, the content about attributes such as “location” and “request type” is spoken. In such cases, if you were using system-led grammars, these "places"
The user's intention regarding "or request type" is not transmitted.

On the other hand, even in the conventional technique, it is possible to always accept the utterance with a high degree of freedom of the user by always using the user-led grammar. However, in this case, the recognition accuracy was low.

In view of the above, the present invention does not limit the range in which the user can speak for each dialogue scene as in the conventional mixed initiative type, and more accurately than the conventional method. It is an object of the present invention to provide a voice interaction device and method capable of grasping the intention of the user.

[0013]

According to the present invention, there is provided voice recognition means for inputting a voice signal of a user and recognizing voice by two different grammars for dialogue to obtain two sets of recognition results. A language understanding unit that analyzes the recognition results of each pair and creates two types of understanding states consisting of item names, values, and certainty of the values corresponding to the recognition results of the two pairs, and the understanding state of the two types of understanding states. The main feature is that a recognition grammar selecting means for deciding which of the two is adopted is provided. The action determining means receives the determined understanding state as an input and determines the next action of the apparatus, and the voice generation / output means outputs the confirmation content or the response content corresponding to the determined behavior as a voice. This is repeated every time the user speaks.

Here, one of the two grammars of the voice recognition means is that, when a question or confirmation is made on the device side before the recognition, an item having the same attribute as the question or confirmation and the dialogue progress The grammar is composed of only general vocabulary necessary for this purpose, and the other grammar is a grammar in which a vocabulary having an attribute different from the attribute of the item that the device has asked or confirmed is included in the recognized vocabulary.

Further, the recognition grammar selecting means compares the reliability of the two sets of recognition results and determines which of the two kinds of understanding states is to be adopted, as a first implementation method. Second
In the realization method, in addition to the reliability of the two sets of recognition results obtained by the speech recognition unit, the reliability of the immediately preceding two sets of recognition results, the elapsed time of the dialogue, the number of dialogue exchanges, and the user's voice. 2 based on some or all of the length, power, and prosody information
Decide which of the different comprehension states to adopt.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a voice interaction device according to the present invention.
The voice interaction device 100 includes a voice input unit 110, a voice information extraction unit 110, a voice recognition unit 130, a language understanding unit 140,
The recognition grammar selection unit 150, the action determination unit 160, the voice generation output unit 170, and the task usage database 180 are included. Further, the voice recognition unit 130 includes a recognition unit (system-driven voice recognition unit) 131 having a system-driven dialogue grammar and a recognition unit (user-driven speech recognition unit) 132 having a user-driven dialogue grammar. Composed. The voice interaction device 100 is actually configured by using a so-called computer.

FIG. 2 is a flow chart showing an example of the processing procedure of the voice dialog device 100. The operation of each unit in FIG. 1 will be described below with reference to FIG.

First, in the voice input section 110, the voice uttered by the user is received as an input and converted into an electric signal,
Further, the converted digital signal is output (step 201). For the voice input unit 110, a general microphone or a combination of a telephone and an AD converter can be used.

Next, the voice information extraction unit 120 receives the voice signal (digital voice signal) output from the voice input unit 110, and outputs information such as the length, power, and prosody of the voice spoken by the user ( Step 202).

On the other hand, the voice recognition unit 130 receives the voice signal (digital voice signal) from the voice input unit 110, and outputs the character string of the recognition result and the reliability (score) of the recognition result (step 203). Since this kind of voice recognition processing itself is well known, its details are omitted.

Here, the voice recognition unit 130 is composed of a recognition unit 131 having a system-initiated dialogue grammar and a recognition unit 132 having a user-initiated dialogue grammar, each of which is a voice signal from the voice input unit 110. To output the character string of the recognition result and the reliability (score) of the recognition result. That is, in the voice recognition unit 130, the voice input unit 1
For the voice signal from 10, two sets of recognition results by the system-driven dialogue grammar and the user-driven dialogue grammar (2
Outputs a set of character strings and their reliability).

For example, assume the following interaction example 3. <Dialogue example 3> Device Utterance 1: "When is the weather forecast?" User utterance 2: "Tomorrow" Device Utterance 3: "Please specify the prefecture name" User utterance 4: "Tomorrow's western Kanagawa prefecture"

In the system driven dialogue grammar, the device utterance 1 is followed by only the date and the day of the week. The system-led dialogue grammar after the device utterance 3 is composed of only the prefecture name. On the other hand, since the user-driven dialogue grammar does not limit the vocabulary according to the scene of the dialogue, the prefecture name and the type of weather are included as the recognition vocabulary even after the device utterance 1, and even after the device utterance 3. , Date and day of the week will also be included.

The vocabulary and grammar of the recognition unit 130 are those that can be exchanged by the instruction of the action determination unit 160 described later.

Next, the language understanding unit 140 receives the character string of the recognition result and the reliability (score) from the voice recognition unit 130 as inputs, and outputs the understanding state (step 20).
4). Here, the understanding state is represented by the item name, the value, and the certainty of the value. The item name is “place”, “time”, “information type”, and the like, and the value is the value of the item name, such as “Kanagawa” or “obvious”. The accuracy of the value is represented by a numerical value, and the reliability obtained from the voice recognition unit 130 can be used. For example, in the weather information guidance system, the understanding state as shown in FIG. 3 is obtained for the recognition result of "Please tell me the temperature of Kanagawa tomorrow."

In the language understanding unit 140, the voice recognition unit 130
The system-led voice recognition unit 131 and the user-led voice recognition unit 132 input two sets of recognition results, and output two types of understanding states as a result of processing. For example, it is assumed that the user's utterance "Tomorrow's Eh" is recognized as "Tomorrow's Ehime's" in the user-driven dialogue grammar and "Tomorrow" in the system-driven dialogue grammar. . In this case, in the language understanding unit 140, as shown in FIG.
Two types of understanding states as shown in FIG. 5 are created and output.

[0027] As another example, the user's utterance is a positive expression such as "yes" or "yes" or "no".
If the expression is a negative expression such as "I'm wrong", "Affirmative or negative" is used as the item name.

Next, in the recognition grammar selecting section 150, the two kinds of understanding states from the language understanding section 140 are input, and the reliability of the two sets of recognition results from the voice recognizing section 130 and the voice information extracting section 120 are input. Based on the user's length, power, prosody, and the time and number of interactions in the task specification database 180 so far, it is determined whether to adopt system-led or user-led, The corresponding understanding state is selected and output (step 205).

Since the conventional speech dialogue system does not have such a recognition grammar selecting section 150, it is necessary to limit the grammar for recognition to one type before the user speaks. , The user's utterance outside the range permitted by the grammar prepared by the device could not be accepted, or the grammar prepared by the device was wider than necessary and the accuracy was low.

Here, two embodiments will be described as a method of realizing the recognition grammar selecting section 150. The first embodiment is system-initiated or user-initiated based on the reliability of two sets of recognition results from the system-initiated speech recognition unit 131 and the user-initiated speech recognition unit 132 of the speech recognition unit 130. Is to determine. Now, the system-led voice recognition unit 13
1. CMs and CMu are obtained by normalizing the reliability (score) output by the user-led voice recognition unit 132.
And If CMs> CMu, the system initiative is adopted, and if CMu> CMs, the user initiative is adopted.
Then, of the two types of understanding states from the language understanding unit 140, the corresponding understanding state is output. FIG. 6 shows a flowchart of this embodiment.

In the second embodiment, in addition to the reliability of the two sets of recognition results from the system-led voice recognition unit 131 and the user-led voice recognition unit 132 of the voice recognition unit 130, the voice information extraction unit 120 is used. The length, power, and prosody of the obtained voice of the user, and for example, the task specification database 1
The two recognition units 13 thus far stored in 80.
Based on the reliability of the output of 1,132, the utterance of the immediately preceding device, the time of the interaction in the previous conversation, the number of interactions in the previous conversation, etc., it is determined whether to be system-initiated or user-initiated. is there. Hereinafter, this embodiment will be described in detail.

In the second embodiment, a machine learning method can be used. As an algorithm for this, a part of the feature amount given as an input, that is, the described understanding state, the reliability of the immediately preceding recognition unit, the reliability of the previous recognition unit, the utterance of the immediately previous device, and the previous conversation Elapsed time, number of dialogue exchanges so far, voice length, power,
The details are not particularly limited as long as it is an algorithm that can classify binary values to be output, that is, system-initiated or user-initiated, when a part (or all) of the prosody is given a feature amount.

Here, as one specific example, the case where the reliability of the immediately preceding user-initiated voice recognition unit, the elapsed time of the dialogue, the number of times of dialogue exchanges, and the length of the voice of the user are used as the feature amount. explain. Collect the user's utterances to the device utterances as learning data. Regarding the utterances of these users, manually mark whether the system-driven grammar allows the dialogue to proceed or not (that is, the user-driven grammar is required). For example, the condition is as follows. Device utterance 1: "When is the weather forecast?" User utterance 2: "Tomorrow" ... System-driven grammar device utterance 3: "Please specify the prefecture name" User utterance 4: "Tomorrow's Kanagawa Prefecture" West ”... User driven grammar

Next, the correspondence between the value of each feature at the time of each user's utterance and the system-initiated grammar marked above or the user-initiated grammar is described as shown in FIG. 7, for example. prepare. When a decision tree is created using these pieces of information, for example, (1) If the number of dialog exchanges is more than 5 ... System-led (2) The number of dialog exchanges is 5 or less, and the reliability is 0.7.
If it is larger ... User-initiated (3) The number of dialogue exchanges is 5 or less, and the reliability is 0.7.
If it is smaller, the elapsed time is 10 seconds or more, and the length of the user's voice is longer than 2 seconds ... System initiative (4) The number of dialogue exchanges is 5 times or less, and the reliability is 0.7.
If it is smaller, the elapsed time is 10 seconds or more, and the length of the user's voice is 2 seconds or less ... User-initiated (5) The number of dialog exchanges is 5 times or less, and the reliability is 0.7.
If it is smaller and the elapsed time is less than 10 seconds, a rule such as "user initiated" can be obtained.

The recognition grammar selecting unit 150 prepares the rule created in this way, applies it every time the user utters, and determines whether it is system-initiated or user-initiated.
Then, of the two types of understanding states from the language understanding unit 140, the corresponding understanding state is selected and output. FIG. 8 shows a processing flowchart corresponding to the above rule.

The action determining section 160 includes a recognition grammar selecting section 15
The understanding state output by 0 is received as an input, and the next action of the device is determined (step 206). Further, the understanding state up to now held in the task specification database 180 is updated, and the utterance content from the device to the user is determined.

The update of the understanding state can be divided into two: integration of item name and value and update of confirmation flag. If the input understanding status does not have the item name "Affirmative or Negative", if the confirmation flag is not set for each item name of the understanding status that has been held until now, the input understanding status Integrate the corresponding parts of. If the input understanding state has the item name "Affirmative or Negative" and the previous output of the action determination unit is confirmation, if affirmative, a confirmation flag of the previously confirmed item is set, If negative, erase the value of the item checked last time.

Further, by comparing the task specification database 180 and the current understanding state, it is confirmed that the current understanding state satisfies all the items necessary for some kind of request that can be accepted by any of the devices, and has been confirmed. If so, it is possible to understand the request contents already,
The response content described in the task specification database 180 is output. If there are unconfirmed items in the current understanding and if they confirm all the necessary information for some kind of request that can be processed by the device, the device will display a confirmation mark and an item. Outputs a list of. If the current understanding state does not satisfy the required items for any request type that the device can accept, the item names are determined in a predetermined order, and the information request mark and the item name are output.

FIG. 9 shows an example of the task specification database 180. The task specification database 180 describes the types of requests that the system can accept, the items required for each type of request, and the response content for each type of request. Although omitted in FIG. 9, the task specification database 180 also holds various kinds of information required by the recognition grammar selecting unit 150 in the second embodiment.

Finally, the voice generation / output unit 170 receives as input one of the utterance content, confirmation, and information request output by the action determination unit 160, and outputs the voice of the device (step 207). When the utterance content includes a mark for retrieving the utterance content, first, the result of the response content search unit search is replaced with the response content. The speech generation / output unit 170 can be realized by a combination of an existing template-based language generator, an existing relational database searcher, and an existing speech synthesizer.

The embodiment of the present invention has been described above. However, some or all of the processing functions of each unit in the apparatus shown in FIG. 1 are configured by a computer program, and the program is executed by the computer. Needless to say, the present invention can be realized, or the processing procedure shown in FIG. 2 can be configured by a computer program and the computer can execute the program. A computer-readable recording medium, such as an FD, MO, or RO, stores a program for realizing the processing function of the computer or a program for causing the computer to execute the processing procedure.
It is possible to record the data on an M, a memory card, a CD, a DVD, a removable disk, etc., and save or provide it, as well as distribute the program through a network such as the Internet.

[0042]

According to the present invention, the grammar according to the initiative is selected after the utterance of the user, so that the user's request content can be grasped more accurately without restricting the utterance of the user. Therefore, it is possible to reduce the trouble of the user.

[0043]

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of a voice interaction device according to the present invention.

FIG. 2 is a processing flowchart of an embodiment of a voice interaction method according to the present invention.

FIG. 3 is an example of an understanding state.

FIG. 4 is an example of an understanding state in which a recognition output of a user-initiated dialogue grammar is processed.

FIG. 5 is an example of an understanding state in which recognition output of a system-driven dialogue grammar is processed.

FIG. 6 is a processing flowchart of a first embodiment of a recognition grammar selecting unit.

FIG. 7 is an example of learning data used in the first embodiment of the recognition grammar selecting unit.

FIG. 8 is a processing flowchart of a second embodiment of a recognition grammar selecting unit.

FIG. 9 is an example of a task specification database.

[Explanation of symbols]

100 voice interaction device 110 Voice input section 120 voice information extraction unit 130 Speech recognition unit 131 System-led voice recognition unit 132 User-Driven Speech Recognition Unit 140 Language Understanding Department 150 Recognition Grammar Selection Section 160 Action decision unit 170 Voice generation output unit 180 task specification database

Continued front page    (72) Inventor Kiyoaki Aikawa             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation F term (reference) 2F029 AA02 AA07 AC02 AC18 AC20                 5D015 AA05 HH15 LL02 LL06 LL09

Claims (10)

[Claims] 1. A voice interactive device for inputting a voice of a user, performing a process according to a request of the user, and outputting a processing result using the voice, a voice input unit for converting a voice into a voice signal of an electric signal. And a voice recognition unit that receives the voice signal and recognizes voices by two different grammars for dialogue to obtain two sets of recognition results, and analyzes each of the two sets of recognition results to obtain an item name and a value. And a grammar selection that determines which of the two types of comprehension states is to be adopted, and two types of comprehension states consisting of A unit, an action determination unit that determines the next action of the device by using the determined understanding state as an input, and a voice generation output unit that outputs confirmation content or response content corresponding to the determined action as a voice, To have Spoken dialogue apparatus according to claim. 2. The voice interaction device according to claim 1,
One of the two grammars of the voice recognition unit is a general one that is required for proceeding with a dialogue when a question or confirmation is made on the device side before recognition, the item having the same attribute as the question or confirmation. Spoken dialogue device characterized in that it is a grammar consisting of only vocabulary, and the other grammar is a grammar in which a vocabulary having an attribute different from the attribute of the item that the device has asked or confirmed is also included in the recognized vocabulary. . 3. The spoken dialogue apparatus according to claim 1 or 2, wherein the recognition grammar selection unit compares the reliability of two sets of recognition results and determines which of two types of understanding states should be adopted. A voice interaction device characterized by: 4. The speech dialogue apparatus according to claim 1, wherein the recognition grammar selecting unit is a speech recognition unit which is obtained by the speech recognizing unit.
In addition to the reliability of the recognition results of the pair, the reliability of the recognition results of the immediately preceding two groups, the elapsed time of the dialogue, the number of times of dialogue exchange, the length of the user's voice, the power, and part or all of the prosodic information. A voice dialogue apparatus characterized by determining which of two kinds of understanding states is to be adopted based on the above. 5. A voice inputting method for converting a voice into a voice signal of an electrical signal in a voice interactive method for receiving a voice of a user as input and performing a process according to a request of the user and outputting a processing result using the voice. And a voice recognition process in which the voice signal is input and voice recognition is performed using two different grammars for dialogue, and two sets of recognition results are obtained. And a grammar selection process that determines which of the two types of understanding states is to be adopted, and two types of comprehension states consisting of A process, an action determination process that inputs the determined understanding state and determines a next action, and a voice generation and output process that outputs confirmation content or response content corresponding to the determined action as voice. You Voice interaction wherein the. 6. The voice interaction method according to claim 5,
The two grammars used in the speech recognition process are, when one asks a question or confirmation on the device side before recognition, the item having the same attribute as the question or confirmation, and
The grammar consists of only general vocabulary necessary for the progress of the dialogue, and the other grammar is a grammar in which the vocabulary of the attribute different from the attribute of the item that the device has asked or confirmed is also included in the recognition vocabulary A spoken dialogue method characterized by being present. 7. The spoken dialogue method according to claim 5, wherein in the recognition grammar selection process, the reliability of two sets of recognition results is compared to determine which of two types of comprehension states should be adopted. A spoken dialogue method characterized by: 8. The spoken dialogue method according to claim 5, wherein in the recognition grammar selection step, in addition to the reliability of the two sets of recognition results obtained in the speech recognition step, the reliability of the immediately preceding two sets of recognition results is added. Determining which of the two comprehension states to employ based on the degree, the elapsed time of the dialogue, the number of dialogue exchanges, the length of the user's voice, power, and some or all of the prosodic information. A spoken dialogue method characterized by. 9. A voice interaction program for executing the voice interaction method according to claim 5, 6, 7 or 8 on a computer. 10. A recording medium recording a voice interaction program for executing the voice interaction method according to claim 5, 6, 7 or 8 on a computer.