JP5818753B2 - Spoken dialogue system and spoken dialogue method - Google Patents

Description

  Embodiments relate to voice interaction.

  2. Description of the Related Art There is known a system that provides a service that meets a user's request through voice dialogue with a user (hereinafter referred to as a voice dialogue system). Conventionally, for example, a certain type of car navigation system provides information desired by a user (for example, route guidance information) through voice interaction. In recent years, the use of such a voice interaction system has been expanded to, for example, applications for mobile phones.

  In the voice dialogue progressed by the voice dialogue system, the voice output of the response sentence from the system and the voice input from the user (that is, the user's speech) are alternately repeated. In general, the longer the response sentence, the longer it takes to complete the voice output of the response sentence. Therefore, in a voice interaction system that does not accept the user's utterance until the voice output of the response sentence is completed, the timing at which the user can start the utterance is limited, so that it is difficult to smoothly advance the voice conversation.

  There has been proposed a voice interaction system (having a so-called barge-in function) that can accept a user's utterance in the middle of outputting a response sentence. In the conventional barge-in function, for example, it is considered that the user's utterance has been made for the entire response sentence including the non-output part. However, due to the fact that the user has only listened to a part of the response sentence, the user may start speaking in a state where the user has misunderstood the meaning of the entire response sentence. In this case, if it is considered that the user's utterance is made with respect to the entire response sentence, the user's utterance and the utterance assumed by the system do not mesh with each other.

  Further, another conventional barge-in function is to process the user's utterance using, for example, the past dialog state after canceling the voice output of the response sentence. However, the user may barge-in, for example, not for rephrasing but for replying to the response sentence being output. In this case, if the user's utterance is processed using the past dialogue state after canceling the voice output of the response sentence, the user's utterance and the utterance assumed by the system do not mesh with each other. Arise.

Japanese translation of PCT publication No. 2002-527829

  The embodiment aims to smoothly advance a voice conversation even when a barge-in from a user occurs.

  According to the embodiment, the voice interaction system includes an output unit, an input unit, a detection unit, an acquisition unit, a first understanding unit, an update unit, a voice recognition unit, a second understanding unit, And a control unit. The output unit outputs a response sentence corresponding to the current dialog state by voice. The input unit inputs a user's utterance. A detection part detects the start timing of a user's utterance, and produces | generates timing information. The acquisition unit acquires a partial response sentence that has been output as a voice at the start timing of the utterance in the response sentence based on the information about the response sentence and the timing information. The first understanding unit obtains one or more first understanding results by understanding the meaning of the partial response sentence. The update unit generates an alternative dialog state for updating the current dialog state based on the one or more first understanding results. The speech recognition unit obtains utterance text by performing speech recognition processing on the utterance. The second understanding unit obtains a second understanding result by understanding the intention of the user's utterance based on at least the utterance text. The control unit updates the current dialog state with the alternative dialog state, processes the second understanding result, and controls the transition between the dialog states.

1 is a block diagram illustrating a voice interaction system according to a first embodiment. The figure which shows the example of advancing of the voice dialogue when the barge-in from a user does not arise. The figure which shows the advancing example of the voice dialogue when the barge-in from a user arises. Explanatory drawing of the barge-in from a user. Explanatory drawing of the influence on the user's utterance intention by the timing of barge-in. The figure which illustrates operation | movement of the voice dialogue system which concerns on 1st Embodiment, and the voice dialogue system which concerns on a comparative example, when the barge-in from a user does not arise. The figure which illustrates operation | movement of the voice interactive system which concerns on a comparative example in case the barge-in from a user arises. The figure which illustrates operation | movement of the speech dialogue system which concerns on 1st Embodiment in case the barge-in from a user arises. The block diagram which illustrates the voice dialogue system concerning a 2nd embodiment. The block diagram which illustrates the voice dialog system concerning a 3rd embodiment. The figure which illustrates operation | movement of the speech dialogue system which concerns on 2nd Embodiment in the case where the barge-in from a user arises. Explanatory drawing of a confirmation sentence. The figure which illustrates operation | movement of the voice interactive system which concerns on a comparative example in case the barge-in from a user arises. Explanatory drawing of operation | movement of the meaning determination part of FIG. FIG. 6 illustrates a technique for generating an alternative interaction state.

Hereinafter, embodiments will be described with reference to the drawings. Hereinafter, the same or similar elements as those already described are denoted by the same or similar reference numerals, and redundant description is basically omitted.
(First embodiment)
If no barge-in occurs from the user (that is, the user speaks after listening to the response sentence from the system), the voice dialogue generally proceeds as shown in FIG. 2, for example. According to the example of FIG. 2, since the user properly understands the response sentence from the system and speaks in response to the response sentence, the utterance is within the system assumption and the voice conversation proceeds smoothly. .

  However, in general, when barge-in occurs from a user, it is not easy to properly understand the user's intention to speak. Specifically, barge-in from the user may occur at various timings as illustrated in FIG. Since the response sentence that the user can hear varies depending on the timing of barge-in, as illustrated in FIG. 5, the user's understanding of the response sentence also varies depending on the timing of barge-in. And since a user utters based on the understanding with respect to a response sentence, even if a user's utterance content is the same as illustrated in FIG. 5, a user's utterance intention may differ depending on the timing of barge-in.

  The voice interaction system according to the first embodiment understands the meaning of a response sentence (hereinafter, referred to as a partial response sentence) heard by the user even when barge-in occurs from the user. The user's utterance is processed using the result. Therefore, according to this voice dialogue system, the voice dialogue can proceed smoothly.

  Specifically, when barge-in occurs from the user (that is, when the user speaks without listening to the response sentence from the system), the voice dialogue proceeds as shown in FIG. 3, for example. In the example of Fig. 3, the user responds with a response from the system (A2) "I don't like Greek government bonds with an annual interest rate of 100%. Then, would you like 7% Italian government bonds or 1% Japanese government bonds? "Yes!" After hearing "Greek government bonds with 100% annual interest rate" (B3). In this case, since the user has not heard the response sentence (A2) from the system to the end, the user may misunderstand the meaning of the received response sentence (C2), and may utter an unexpected utterance of the system. . As will be described later, the voice interaction system according to the present embodiment extracts a partial response sentence from a response sentence based on the start timing of the user's utterance, understands the meaning of the sentence, and uses the sentence understanding result. Update the conversation state. Therefore, according to this spoken dialogue system, it is possible to appropriately handle utterances that are not anticipated by the system.

  As shown in FIG. 1, the voice dialogue system according to the present embodiment includes a voice input unit 101, a voice recognition unit 102, an utterance intention understanding unit 103, a dialogue state control unit 104, and a dialogue history storage unit 105. A voice output unit 106, an utterance detection unit 107, a partial response sentence acquisition unit 108, a partial response sentence understanding unit 109, and a dialog state update unit 110.

  The voice interaction system of FIG. 1 may be implemented by a single device or may be implemented across two or more devices. For example, some elements of the voice interaction system of FIG. 1 are provided in a server device, and the remaining elements are connected to an electronic device (for example, a mobile phone, a PC, a portable media player, etc.) that can be connected to the server device via a network. May be provided.

  The voice input unit 101 may be implemented using a microphone, for example. In this case, the voice input unit 101 can obtain a speech signal by converting voice input from the user into an electrical signal. Alternatively, the voice input unit 101 may receive a speech signal from a device not shown in FIG. 1 via a network. The voice input unit 101 outputs the utterance signal to the voice recognition unit 102 and the utterance detection unit 107. When the voice input unit 101 is provided in a device different from the voice recognition unit 102 and the utterance detection unit 107, the voice input unit 101 may output (send) an utterance signal via a network.

  The voice recognition unit 102 inputs an utterance signal from the voice input unit 101. The speech recognition unit 102 performs speech recognition processing on the speech signal, thereby obtaining speech text representing the content of the user's speech. The speech recognition unit 102 may implement speech recognition processing using any technique.

  The utterance intention understanding unit 103 inputs the utterance text from the voice recognition unit 102. The speech intention understanding unit 103 obtains a speech intention understanding result by understanding the user's speech intention based on the speech text. Specifically, the utterance intention understanding unit 103 may understand the intention of the user's utterance based on the utterance text by using the same or similar technique as the partial response sentence understanding unit 109 described later. Here, the utterance intention understanding result means data that abstracts and expresses the intention of the utterance text. The utterance intention understanding unit 103 outputs the utterance intention understanding result to the dialog state control unit 104.

  If the utterance text is ambiguous (for example, “Yes”, “No”, “Yes”, etc.), it is difficult to specify the intention of the user's utterance with high accuracy from the utterance text alone. Therefore, the utterance intention understanding unit 103 understands the intention of the user's utterance based on the dialogue history from the dialogue history storage unit 105 and the meaning of the partial response sentence from the partial response sentence understanding unit 109 in addition to the utterance text. May be. The utterance intention understanding unit 103 can understand the intention of the user's utterance more accurately by considering at least one of the dialogue history and the sentence understanding result.

  The dialogue state control unit 104 inputs an utterance intention understanding result from the utterance intention understanding unit 103. The dialogue state control unit 104 processes the utterance intention understanding result using the current dialogue state, and controls transitions between dialogue states (including self-transitions) as necessary. Specifically, the voice dialogue system in FIG. 1 prepares a dialogue state for each scenario, and realizes a voice dialogue with the user by performing a transition between dialogue states according to the utterance intention understanding result.

  Here, the dialogue state is a parameter that determines the behavior of the voice dialogue system. The dialogue state includes, for example, a response sentence that the voice dialogue system outputs to the user, a word that the voice dialogue system can accept for the response sentence, and an operation of the voice dialogue system when the acceptable word is input. Etc.

  The dialogue state control unit 104 determines the response sentence to be output next to the user by processing the utterance intention understanding result. The dialog state control unit 104 outputs a response sentence to the voice output unit 106. Furthermore, the dialogue state control unit 104 outputs the current dialogue state and the utterance intention understanding result to the dialogue history storage unit 105 as a dialogue history.

  Furthermore, the dialogue state control unit 104 can input an alternative dialogue state from the dialogue state update unit 110 described later. The dialog state control unit 104 processes the utterance intention understanding result after updating the current dialog state with the alternative dialog state.

  The dialogue history storage unit 105 stores the dialogue history from the dialogue state control unit 104. The dialogue history stored in the dialogue history storage unit 105 is read by the utterance intention understanding unit 103 as necessary.

  The audio output unit 106 may be mounted using a speaker, for example. In this case, the voice output unit 106 can output the response sentence from the dialogue state control unit 104 as a voice. When the response sentence is in a text format, the voice output unit 106 may output the response sentence by voice by performing, for example, an arbitrary text reading process. Alternatively, when the response sentence is in a voice format, the voice output unit 106 may output the response sentence by performing necessary signal processing. Or the audio | voice output part 106 may transmit a response sentence via a network to the apparatus which is not shown by FIG. When the voice output unit 106 is provided in a device different from the dialog state control unit 104, the voice output unit 106 may input (receive) a response sentence via a network.

  Further, the voice output unit 106 outputs information about the response sentence to the partial response sentence acquisition unit 108. The information regarding the response sentence includes, for example, character string information indicating a character string constituting the response sentence, and time information indicating a voice output time of each character constituting the character string. When the voice output unit 106 is provided in a device different from the partial response text acquisition unit 108, the voice output unit 106 may output (send) information about the response text via the network.

  The utterance detection unit 107 inputs an utterance signal from the voice input unit 101. The utterance detection unit 107 detects the start timing of the user's utterance based on the utterance signal. The start timing means, for example, the time when the utterance is started, the number of characters of the response sentence that has already been output at the time when the utterance is started, and the like. The utterance detection unit 107 generates timing information using the detected start timing. The utterance detection unit 107 outputs the timing information to the partial response sentence acquisition unit 108. When the utterance detection unit 107 is provided in a device different from the voice input unit 101, the utterance detection unit 107 may input (receive) an utterance signal via a network. When the utterance detection unit 107 is provided in a device different from the partial response sentence acquisition unit 108, the utterance detection unit 107 may output (transmit) timing information via a network.

  The timing information may specify the start timing itself, or may specify the timing obtained by correcting to the start timing. In general, a certain amount of delay occurs until a human hears a voice, understands the content of the voice, and speaks based on the understood content. Therefore, the utterance detection unit 107 may generate timing information that identifies a timing when the user's utterance start timing is traced back by a correction amount (time or number of characters) corresponding to a delay caused by the user's comprehension ability. Further, the utterance detection unit 107 may consider other delays such as network transmission. Note that the correction of the start timing in consideration of the delay may be performed in the partial response sentence acquisition unit 108.

  The partial response sentence acquisition unit 108 inputs information about the response sentence from the voice output unit 106 and inputs timing information from the utterance detection unit 107. The partial response text acquisition unit 108 extracts a partial response text from the entire response text based on the information about the response text and the timing information. The partial response text acquisition unit 108 outputs the partial response text to the partial response text understanding unit 109.

  The last character of the partial response sentence is determined by the timing information. For example, if the timing information represents a time, the last character output by voice up to that time may be determined as the last character of the partial response sentence. Further, if the timing information represents the number of characters, a character that is the number of characters ahead of the first character of the response sentence may be determined as the last character of the partial response sentence.

  Typically, the first character of the response sentence may be determined as the first character of the partial response sentence. Alternatively, when the response sentence includes a plurality of sentences, the first character of any of the plurality of sentences may be determined as the first character of the partial response sentence. When the response sentence includes a plurality of topics, the first character of the partial response sentence may be determined based on the turning point of the topic.

  The partial response text understanding unit 109 inputs the partial response text from the partial response text acquisition unit 108. The partial response sentence understanding unit 109 obtains the meaning understanding result of the partial response sentence by understanding the meaning of the partial response sentence. Here, the meaning understanding result of the partial response sentence means data that abstractly expresses the meaning of the partial response sentence. It should be noted that the partial response sentence understanding unit 109 automatically determines any one sentence meaning understanding result even when a plurality of sentences are expected in the partial response sentence. The partial response sentence understanding unit 109 outputs the sentence understanding result to the utterance intention understanding unit 103 and the dialog state updating unit 110.

  For example, the partial response sentence understanding unit 109 may convert the partial response sentence into a tag that abstractly expresses the meaning of the partial response sentence. For example, in the following description, the meaning of the partial response sentence is expressed by a combination of tags called “act” and “command”. However, the meaning of the partial response sentence may be expressed by only the action tag or one or more other tags. Alternatively, the partial response sentence understanding unit 109 may convert the partial response sentence into any of the example sentences listed in the example sentence collection in which a large number of sentences are arranged and normalized in advance.

  The partial response sentence understanding unit 109 may determine the meaning of the partial response sentence according to a rule in which the correspondence between the partial response sentence and the meaning of the sentence is determined. Alternatively, the partial response sentence understanding unit 109 calculates a probability that a partial response sentence and an arbitrary sentence meaning are associated with each other based on a statistical learning result using a large amount of learning data, and a sentence showing the highest probability. You may choose your will.

  The dialogue state update unit 110 inputs a sentence understanding result from the partial response sentence understanding unit 109. The dialog state update unit 110 obtains an alternative dialog state for updating the current dialog state by generating a dialog state corresponding to the sentence understanding result. The dialog state update unit 110 outputs the alternative dialog state to the dialog state control unit 104.

  Hereinafter, the operation of the voice interaction system of FIG. 1 is compared with the operation of the voice interaction system according to the comparative example with reference to FIGS. 6, 7, 8, and 15. Here, it is assumed that the voice interaction system according to the comparative example assumes that the user's utterance has been made with respect to the entire response sentence including the unoutput portion when barge-in occurs from the user.

  When barge-in from the user does not occur, the voice interaction system of FIG. 1 and the voice interaction system according to the comparative example operate as illustrated in FIG. 6, 7, 8, 11, and 13, the user's utterance intention understood by the system is the subject (“User”; first line), the action tag (second line), and the command tag. (3rd line). Similarly, in FIG. 6, FIG. 7, FIG. 8, FIG. 11 and FIG. 13, the meaning of the response sentence is the subject ("System"; the first line), the action tag (second line), and the command tag (third line). Eyes).

  According to the example of FIG. 6, the user can listen to the response sentence to the end and appropriately understand the meaning of the entire response sentence. Therefore, the utterance from the user is within the assumption of the system, and the voice conversation proceeds smoothly.

  When barge-in from the user occurs, the voice interaction system according to the comparative example operates as shown in FIG. 7, for example. In the example of Fig. 7, the user responds with a response from the system (A2) "I don't like Greek government bonds with an annual interest rate of 100%. Then, would you like 7% Italian government bonds or 1% Japanese government bonds?" Among them, “Yes!” Was spoken after hearing “Greek government bonds with 100% annual interest rate” (B3). In this case, since the user has not heard the response sentence (A2) from the system to the end, he / she misunderstood the meaning of the sentence of the answer sentence heard, for example, “Do you want to make Greek government bonds?”. That is, the intention of the user's utterance (B3) is, for example, “I want to purchase a Greek government bond with an interest rate of 100%!” (D3).

  On the other hand, the system assumes that the user understands the overall meaning of the response sentence (A2). Therefore, the dialog state (D2) of the system assumes an utterance corresponding to the country name “Italy” or the country name “Japan”. As a result, the user's utterance (B3) is treated as an unexpected system, and there is no “Yes” country. Please enter your country name. A response sentence (A4) like “ This response sentence (A4) is not appropriate in view of the utterance intention (D3) of the user who wishes to purchase Greek government bonds. Therefore, the progress of the voice dialogue is hindered.

  When barge-in occurs from the user, the voice interaction system of FIG. 1 operates as shown in FIG. 8, for example. In the example of Fig. 8, the user responds with a response from the system (A2) "I don't like Greek government bonds with an annual interest rate of 100%. Then, would you like 7% Italian government bonds or 1% Japanese government bonds?" Among them, “Yes!” Was spoken after hearing “Greek government bonds with 100% annual interest rate” (B3-1). In this case, since the user has not heard the response sentence (A2) from the system to the end, he / she misunderstood the meaning of the sentence of the answer sentence heard, for example, “Do you want to make Greek government bonds?”.

  The system detects the user's utterance (B3-1), and acquires the partial response sentence “Greek government bond with 100% annual interest rate” (A3-2). The system understands the meaning (C3-2) of the partial response sentence (A3-2), generates an alternative dialog state (D3-2), and updates the current dialog state. Therefore, the dialogue state of the system assumes an utterance corresponding to affirmation or denial. As a result, the user's utterances (B3-1, B3-3) are treated as being within the assumption of the system, and it is “an investment in Greek government bonds. I understand. How much will you invest?” (A4 ) Is output by voice. Since this response sentence (A4) is appropriate in view of the utterance intentions (D3-1, D3-3) of the user who wishes to purchase the Greek government bond, the user is, for example, “10,000 for the time being” (B5). Spoken and voice conversation proceeds smoothly.

  By the way, in order to generate the alternative dialogue state according to the sentence understanding result of the partial response sentence, when outputting the sentence meaning of the partial response sentence and the response sentence corresponding to the sentence meaning as illustrated in FIG. It is possible to create in advance and use information that is a combination of the dialog state (that is, the alternative dialog state). The table illustrated in FIG. 15 represents a set of a partial response sentence (column B), its sentence meaning (column C), and a dialogue state (column D) when outputting a response sentence corresponding to the sentence meaning. Yes. Further, as illustrated in FIG. 15, the previous dialog state information (column A) may be added as an element of the set. The information indicating the previous conversation state (column A) is not essential for generating the alternative conversation state, but more appropriate meaning understanding results when a plurality of meaning understanding results are obtained from the partial response sentence. May be used to identify.

  For example, two sentence meaning understanding results (C1 and C2 in FIG. 15 can be obtained from the partial response sentence (A3-2 in FIG. 8). However, according to the example in FIG. 15, the previous dialog state is “Request. , “Recommend (concountry)” (C1 in FIG. 8), the immediately previous dialog state corresponds to the “interactive state other than“ S1 (= Request, BoundInfo (Greece)) ”(ie, A1 in FIG. 15). The sentence understanding result of the partial response sentence (A3-2 in FIG. 8) is specified as “Question-YN, purchase of government bonds (Greece)” (C1 in FIG. 15).

  More specifically, the combination of the partial response sentence (A3-2 in FIG. 8) and the previous dialog state (C1 in FIG. 8) “Request, Recommend (country)” results in the understanding of the meaning of the partial response sentence (see FIG. Since 15 C1) is specified, an alternative dialogue state (D1 in FIG. 15) according to the sentence understanding result can be generated. Then, the voice dialogue system can smoothly advance the voice dialogue as described above by processing the user's utterance after replacing the dialogue state as (D3-2 in FIG. 8).

  As described above, the voice interaction system according to the first embodiment acquires a partial response sentence based on the start timing of the user's utterance, understands the meaning of the partial response sentence, and updates the conversation state. . Therefore, according to this spoken dialogue system, the dialogue state is updated in conformity with the meaning understanding result of the partial response sentence, so that the user's utterance is easily handled as assumed in the system. That is, according to this voice dialogue system, voice dialogue can proceed smoothly even when barge-in occurs from the user.

(Second Embodiment)
As shown in FIG. 9, the voice dialogue system according to the second embodiment includes a voice input unit 101, a voice recognition unit 102, an utterance intention understanding unit 103, a dialogue state control unit 104, and a dialogue history storage unit. 105, a voice output unit 106, an utterance detection unit 107, a partial response text acquisition unit 108, a partial response text understanding unit 209, a dialog state update unit 110, and a text verification unit 211.

  The partial response text understanding unit 209 inputs the partial response text from the partial response text acquisition unit 108. The partial response sentence understanding unit 209 obtains one or more sentence meaning understanding results by understanding the meaning of the partial response sentence. In other words, the partial response sentence understanding unit 209 may obtain a plurality of meaning understanding results when a plurality of meanings are expected in the partial response sentence. The partial response sentence understanding unit 209 can use the same or similar technique as the partial response sentence understanding unit 109 described above in order to obtain one or more sentence understanding results. The partial response sentence understanding unit 209 outputs one or more sentence understanding results to the sentence confirmation unit 211.

  The text confirmation unit 211 inputs one or more textual understanding results from the partial response text understanding unit 209. The sentence confirmation unit 211 generates one or more confirmation sentences by converting one or more sentence understanding results. In addition, when the number of the text meaning understanding results input from the partial response text understanding section 209 is one, the text meaning confirmation section 211 outputs the text meaning understanding results to the dialog state update section 110 as they are. Also good. Here, the confirmation sentence means text that assists the user's understanding of the contents of the corresponding sentence understanding result. Specifically, as illustrated in FIG. 12, when the understanding result of the partial response sentence is “Question-YN, JGB purchase (Greece)” (B1), the understanding result of the sentence is presented as it is. Instead, by presenting a confirmation sentence such as “How is the Greek government bond?” (C1), it is possible to assist the user in understanding the contents of the meaning understanding result.

  The sentence confirmation unit 211 presents one or more confirmation sentences to the user. The sentence confirmation unit 211 may present one or more confirmation sentences in an arbitrary manner. In general, a video can present a sentence in a shorter time than a voice. Therefore, from the viewpoint of facilitating a voice dialogue, the text confirmation unit 211 does not illustrate a video in which one or more confirmation texts are listed, for example. You may display on a display part. Of course, the sentence confirmation unit 211 may present one or more confirmation sentences by voice, or may present both video and voice.

  According to the example of FIG. 12, when the partial response sentence is “Greek government bond with 100% annual interest rate” (A1, A2), two sentence understanding results (B1, B2) are obtained. Therefore, the text confirmation unit 211 presents to the user confirmation texts corresponding to these, “How about Greek government bonds?” And “Greek government bonds have an annual interest rate of 100%.”

  The user selects one of the presented confirmation sentences that best matches the understanding of the response sentence that he / she has heard. Then, the sentence confirmation unit 211 accepts the user's selection for any one of the one or more confirmation sentences. The text confirmation unit 211 may accept the user's selection in an arbitrary manner. Specifically, the text confirmation unit 211 may accept a user's selection in the form of voice input, touch panel input, key input, gesture input, or the like. The textual intention confirmation unit 211 outputs one textual meaning understanding result for one confirmation text selected by the user to the dialog state update unit 110.

  The sentence confirmation unit 211 may accept an input indicating that none of the presented one or more confirmation sentences matches the understanding of the response sentence heard by the user. When the sentence verification unit 211 accepts the input, a response sentence corresponding to the current conversation state may be output again, or a process related to the partial response sentence (for example, a process of acquiring a partial response sentence, The process of understanding the meaning of the partial response sentence, etc.) may be redone.

  In addition to the confirmation sentence corresponding to the meaning understanding result of the partial response sentence, the intention confirmation unit 211 may present a confirmation sentence (or the response sentence itself) corresponding to the meaning of the entire response sentence to the user. . For example, when the user is accustomed to using the voice interaction system of FIG. 9, it is assumed that the user understands the meaning of the entire response sentence when the user hears a part of the response sentence. However, depending on the start timing of the user's utterance, none of the confirmation sentences corresponding to the meaning understanding result of the partial response sentence may correspond to the meaning of the entire response sentence. That is, depending on the start timing of the user's utterance, there is a possibility that an appropriate confirmation sentence may not be presented. Therefore, it is useful to smoothly advance the voice conversation in this use case by presenting the confirmation sentence corresponding to the meaning of the entire response sentence regardless of the start timing of the user's utterance.

  According to the meaning verification unit 211, even if there are a plurality of meaning-understanding results, one appropriate meaning-understanding result can be extracted based on the user's selection. Therefore, according to the text meaning confirmation unit 211, it is possible to prevent a situation in which the progress of the voice conversation is hindered by the dialog state being updated based on the text understanding result different from the user's understanding.

  The dialog state update unit 110 inputs a text understanding result from the text confirmation unit 211. The dialog state update unit 110 obtains an alternative dialog state for updating the current dialog state by generating a dialog state corresponding to the sentence understanding result. The dialog state update unit 110 outputs the alternative dialog state to the dialog state control unit 104.

  Hereinafter, the operation of the voice interaction system of FIG. 9 is compared with the operation of the voice interaction system according to the comparative example with reference to FIGS. 11 and 13. Here, it is assumed that the voice interaction system according to the comparative example assumes that the user's utterance has been made with respect to the entire response sentence including the unoutput portion when barge-in occurs from the user.

  When barge-in occurs from the user, the voice interaction system according to the comparative example operates as shown in FIG. 11, for example. In the example of Fig. 11, the user responds with a response from the system (A2) "I don't like Greek government bonds with an annual interest rate of 100%. Then, would you like 7% Italian government bonds or 1% Japanese government bonds?" Among them, after hearing “Greek bonds with 100% annual interest rate”, he said, “Why?” (B3). In this case, since the user has not heard the response sentence (A2) from the system to the end, he misunderstood the meaning of the sentence of the response sentence that he heard, such as “Greek government bond is 100% annual interest rate”. . That is, the intention of the user's utterance (B3) is, for example, “request for explanation of the situation of Greek government bonds” (D3).

  On the other hand, the system assumes that the user understands the overall meaning of the response sentence (A2). Therefore, the dialog state (D2) of the system assumes an utterance corresponding to the country name “Italy” or the country name “Japan”. As a result, the user's utterance (B3) is treated as an unexpected one of the system, and there is no country called “Why”. Please enter your country name. A response sentence (A4) like “ This response sentence (A4) is not appropriate in view of the utterance intention (D3) of the user who wishes to explain the situation of the Greek government bond. Therefore, the progress of the voice dialogue is hindered.

  When barge-in occurs from the user, the voice interaction system of FIG. 9 operates as shown in FIG. 13, for example. In the example of Fig. 13, the user responds with a response from the system (A2) "I hate Greek government bonds with an annual interest rate of 100%. Then, would you like 7% Italian government bonds or 1% Japanese government bonds?" Among them, after hearing "Greek government bonds with 100% annual interest rate", he said, "Why?" (B3-1). In this case, since the user has not heard the response sentence (A2) from the system to the end, the sentence of the response sentence heard is, for example, "How about the Greek government bond?" ”And misunderstood.

  The system detects the user's utterance (B3-1), and acquires the partial response sentence “Greek government bond with 100% annual interest rate” (A3-2). The system understands the meaning of the partial response sentence (A3-2) and obtains a plurality of sentence meaning understanding results (for example, B1 and B2 in FIG. 12). The system obtains a plurality of confirmation sentences (for example, C1 and C2 in FIG. 12) by converting a plurality of sentence meaning understanding results. The system presents a plurality of confirmation sentences and accepts user selections for these. Then, the system extracts one sentence meaning understanding result (C3-2) corresponding to the user's selection, generates an alternative dialogue state (D3-2) based on the result, and updates the current dialogue state.

  Therefore, the user's utterances (B3-1, B3-3) are treated as being within the system's assumptions, as “Greece may become the default in the national financial debt crisis” (A4) Such a response sentence is output as voice. Since this response sentence (A4) is appropriate in view of the utterance intentions (D3-1, D3-3) of the user who wishes to explain the situation of the Greek government bond, the voice dialogue proceeds smoothly.

  As described above, the voice interaction system according to the second embodiment presents a confirmation sentence corresponding to one or more meaning understanding results of the partial response sentence to the user, accepts the user's selection, and selects the user. The dialogue state is updated based on one sentence understanding result corresponding to. Therefore, according to this voice dialogue system, it is possible to prevent a situation in which the progress of the voice dialogue is hindered by updating the dialogue state on the basis of the sentence understanding result different from the user's understanding.

(Third embodiment)
As shown in FIG. 10, the voice dialogue system according to the third embodiment includes a voice input unit 101, a voice recognition unit 102, an utterance intention understanding unit 303, a dialogue state control unit 104, and a dialogue history storage unit. 105, a voice output unit 106, an utterance detection unit 107, a partial response sentence acquisition unit 108, a partial response sentence understanding unit 209, a dialog state update unit 110, and a sentence determination unit 312.

  The utterance intention understanding unit 303 inputs the utterance text from the voice recognition unit 102. The utterance intention understanding unit 303 obtains a first utterance intention understanding result by understanding the user's utterance intention based on the utterance text. Specifically, the utterance intention understanding unit 303 may understand the user's utterance intention based on the utterance text by using the same or similar technique as the utterance intention understanding unit 103. The utterance intention understanding unit 303 outputs the first utterance intention understanding result to the sentence determination unit 312. Further, the utterance intention understanding unit 303 may output the first utterance intention understanding result to the dialogue state control unit 104.

  If the utterance text is ambiguous (for example, “Yes”, “No”, “Yes”, etc.), it is difficult to specify the intention of the user's utterance with high accuracy from the utterance text alone. Therefore, the utterance intention understanding unit 103 may understand the intention of the user's utterance based on the dialogue history from the dialogue history storage unit 105 in addition to the utterance text. The utterance intention understanding unit 103 can understand the user's utterance intention more accurately by considering the dialogue history.

  The utterance intention understanding unit 303 may input a sentence understanding result from the sentence intention determination unit 312 described later. Then, the utterance intention understanding unit 303 may generate a second utterance intention understanding result based on the sentence understanding result in addition to the utterance text (and the conversation history). The utterance intention understanding unit 303 can understand the intention of the user's utterance more accurately by considering the sentence understanding result. The utterance intention understanding unit 303 may output the second utterance intention understanding result to the dialogue control unit 104 instead of the first utterance intention understanding result.

  The dialogue state control unit 104 inputs the first utterance intention understanding result or the second utterance intention understanding result from the utterance intention understanding unit 103. The dialogue state control unit 104 processes the first utterance intention understanding result or the second utterance intention understanding result using the current dialogue state, and controls transitions (including self-transition) between the dialogue states as necessary. To do.

  The sentence determination unit 312 inputs the first utterance intention understanding result from the utterance intention understanding unit 303, and inputs one or more sentence intention understanding results from the partial response sentence understanding unit 209. As will be described later, the sentence determination unit 312 determines one of the one or more sentence understanding results with the highest reliability. In addition, when the number of the text meaning understanding results input from the partial response text understanding section 209 is one, the text meaning determination section 312 outputs the text meaning understanding result to the dialog state update section 110 as it is. Also good. The meaning determination unit 312 outputs the result of understanding of the meaning with the highest reliability to the dialog state update unit 110.

  Specifically, the sentence determination unit 312 can refer to a first probability that each of one or more sentence understanding results corresponding to the partial response sentence appears as illustrated in FIG. The first probability corresponds to the probability that the target sentence understanding result matches the user's understanding of the partial response sentence when a partial response sentence as a premise is given. According to the example of FIG. 14, when the partial response sentence is “Greek government bond with 100% annual interest rate” (A1, A2), the probability that each of the two sentence understanding results (B1, B2) appears is 70. % (C1) and 30% (C2).

  Further, as illustrated in FIG. 14, the sentence intention determination unit 312 causes each of a plurality of utterance intentions of the user to appear when each of one or more meaning understanding results corresponding to the partial response sentence is assumed to appear. A second probability can be referenced. The second probability corresponds to the probability that the user utters corresponding to the target utterance intention when the understanding result of the partial response sentence as a premise matches the user's understanding of the partial response sentence. That is, if this second probability is used, it is possible to estimate the understanding of the partial response sentence of the user, which is the premise, from the first utterance intention understanding result. According to the example of FIG. 14, when the appearance of the sentence understanding result (B1) “Question-YN, purchase of government bonds (Greece)” is assumed, the probability that the user speaks corresponding to “Response” (D1). Is “80%” (E1).

  The sentence determination unit 312 determines a first probability corresponding to a certain meaning understanding result and a plurality of second probabilities corresponding to the meaning understanding result that match the first utterance intention understanding result. By multiplying, the reliability of the meaning understanding result is evaluated.

  In the example of FIG. 14, it is assumed that the partial response sentence is “Greek government bond with 100% annual interest rate” (A1, A2), and the first utterance intention understanding result is “Request”. In this case, the first probability corresponding to the sentence understanding result (B1) “Question-YN, purchase of government bonds (Greece)” is “70%” (C1), and the sentence understanding result (B1) Of the plurality of corresponding second probabilities (E1), the one that matches the first utterance intention understanding result is “5%”. Therefore, the reliability of the sentence understanding result (B1) can be evaluated as “3.5%”, for example. In addition, the first probability corresponding to the sentence understanding result (B2) of “Inform, government bond information (Greece)” is “30%” (C2), and there are a plurality of correspondences corresponding to the sentence understanding result (B2). Of the second probabilities (E2), the one that matches the first utterance intention understanding result is “65%”. Therefore, the reliability of the sentence understanding result (B2) can be evaluated as, for example, “19.5%”. The sentence determination unit 312 determines one of the plurality of sentence understanding results (B1, B2) having the highest reliability (B2).

  According to the sentence determination unit 312, even when there are a plurality of sentence understanding results, the sentence determination understanding result having the highest reliability is automatically determined using the first utterance intention understanding result. be able to. Therefore, according to the sentence determination unit 312, a sentence understanding result that matches the user's understanding of the partial response sentence can be determined with high accuracy and in a short time, so that the voice conversation can proceed smoothly.

  The dialog state update unit 110 inputs a text understanding result from the text determination unit 312. The dialog state update unit 110 obtains an alternative dialog state for updating the current dialog state by generating a dialog state corresponding to the sentence understanding result. The dialog state update unit 110 outputs the alternative dialog state to the dialog state control unit 104.

  As described above, the speech dialogue system according to the third embodiment automatically uses one of the one or more sentence meaning understanding results of the partial response sentence with the highest reliability using the user's utterance intention understanding result. The dialog state is updated based on the sentence understanding result having the highest reliability. Therefore, according to the voice dialogue system, the sentence understanding result that matches the user's understanding of the partial response sentence can be determined with high accuracy and in a short time, so that the voice dialogue can proceed smoothly.

  The processing of each of the above embodiments can be realized by using a general-purpose computer as basic hardware. The program for realizing the processing of each of the above embodiments may be provided by being stored in a computer-readable storage medium. The program is stored in the storage medium as an installable file or an executable file. Examples of the storage medium include a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), and a semiconductor memory. The storage medium may be any as long as it can store the program and can be read by the computer. Further, the program for realizing the processing of each of the above embodiments may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 101 ... Voice input part 102 ... Voice recognition part 103,303 ... Utterance intention understanding part 104 ... Dialogue state control part 105 ... Dialogue history memory | storage part 106 ... Voice output part 107 ...・ Speech detection unit 108... Partial response sentence acquisition unit 109, 209... Partial response sentence understanding unit 110... Dialogue state update unit 211.

Claims (10)

An output unit that outputs a response sentence corresponding to the current dialogue state by voice;
An input unit for inputting the user's utterance;
A detection unit that detects a start timing of the user's utterance and generates timing information;
Based on the information related to the response sentence and the timing information, an acquisition unit that acquires a partial response sentence that has been output as a voice at the start timing of the utterance of the response sentence;
A first understanding unit that obtains one or more first understanding results by understanding the meaning of the partial response sentence;
An update unit for generating an alternative dialogue state for updating the current dialogue state based on the one or more first understanding results;
A speech recognition unit that obtains speech text by performing speech recognition processing on the speech;
A second understanding unit for obtaining a second understanding result by understanding the intention of the user's utterance based on at least the utterance text;
A voice dialog system comprising: a controller that updates the current dialog state with the alternative dialog state, processes the second understanding result, and controls transition between the dialog states.   The second understanding unit obtains the second understanding result by understanding an intention of the user's utterance based on the one or more first understanding results in addition to the utterance text. Voice dialogue system.   The spoken dialogue system according to claim 1, wherein the timing information is time information based on the start timing.   The spoken dialogue system according to claim 1, wherein the timing information is character number information based on the start timing.   The voice interaction system according to claim 1, wherein the timing information specifies a timing that goes back a correction amount corresponding to a delay caused by the understanding ability of the user from the start timing.   The spoken dialogue system according to claim 1, wherein the one or more first understanding results are expressed in a tag format. The one or more first understanding results are converted into one or more confirmation sentences, the one or more confirmation sentences are presented to the user, and the user's selection for any one of the one or more confirmation sentences is accepted. Further comprising a confirmation unit
The update unit generates the alternative dialogue state based on one of the one or more first understanding results corresponding to the user's selection;
The speech dialogue system according to claim 1. Assuming the first probability that each of the one or more first understanding results appears and the appearance of each of the one or more first understanding results, an utterance corresponding to the second understanding result appears. A determination unit that evaluates the reliability of each of the one or more first understanding results based on the second probability and determines one of the one or more first understanding results having the highest reliability. Further comprising
The update unit generates the alternative conversation state based on the highest reliability among the one or more first understanding results.
The speech dialogue system according to claim 1. Outputting a response sentence corresponding to the current dialog state,
Enter the user ’s utterance,
Obtaining timing information by detecting the start timing of the user's utterance;
Based on the information on the response sentence and the timing information, obtaining a partial response sentence that has already been voice output at the start time of the utterance of the response sentence;
Obtaining one or more first understanding results by understanding the meaning of the partial response sentence;
Generating an alternative interaction state for updating the current interaction state based on the one or more first understanding results;
Obtaining speech text by performing speech recognition processing on the speech;
Obtaining a second understanding result by understanding the intention of the user's utterance based at least on the utterance text;
Updating the current dialog state with the alternative dialog state, processing the second understanding result, and controlling a transition between the dialog states. Means for outputting a response sentence corresponding to the current dialog state to the computer,
Means to input user utterances,
Means for obtaining timing information by detecting a start timing of the user's utterance;
Based on the information about the response sentence and the timing information, a means for acquiring a partial response sentence that has already been voice output at the start timing of the utterance out of the response sentence;
Means for obtaining one or more first understanding results by understanding the meaning of the partial response sentence;
Means for generating an alternative interaction state for updating the current interaction state based on the one or more first understanding results;
Means for obtaining speech text by performing speech recognition processing on the speech;
Means for obtaining a second understanding result by understanding an intention of the user's utterance based at least on the utterance text;
A voice dialogue program for processing the second understanding result after updating the current dialogue state with the alternative dialogue state and functioning as a means for controlling transition between the dialogue states.

Images