JP2015118710A - Conversation support device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support conversation when speakers talk in different languages or the same language, by presenting information to supplement knowledge according to the conversation.SOLUTION: According to an embodiment, a conversation support device comprises: an input part; a voice recognition part; a conversation history database; an estimation part; a determination part; a creation part; a selection part; and a presentation part. The input part inputs a voice in conversation between speakers. The voice recognition part recognizes the input voice and converts it into corresponding text information. The conversation history database stores all or a part of the text information as a conversation history. The estimation part estimates speaking action on the basis of the text information. The determination part determines whether supplementary information is presented on the basis of the estimated speaking action. The creation part creates candidates of the supplementary information when determining that the supplementary information is presented. The selection part selects one candidate to be presented among candidates of the supplementary information by using the conversation history. The presentation part presents the selected supplementary information.

Description

  Embodiments described herein relate generally to a dialogue support apparatus, method, and program.

  In recent years, voice-based machine translators have been introduced, and it has become possible to talk with a partner in an unknown language. However, as long as the input information is correctly translated, the other speaker does not always have the same understanding as the original speaker that issued the information. For example, some names (proprietary expressions) such as place names or cooking names are generally well known in the original speaker's cultural area or country, but in the other speaker's cultural area, etc. There are many things that are not well known in general, and even if the name given by the original speaker is translated into a translation, the other speaker understands the translation because there is no knowledge There are things that cannot be done.

  Also, even if a person who understands a name, etc. (for example, a person in a cultural area where the name is common) intends to explain it carefully, background knowledge varies greatly among speakers. Therefore, the same image may not be transmitted. For example, when you ask for an explanation because the contents are not known from the name of the dish in the restaurant, the image may actually be different due to differences in prerequisite knowledge such as different general cooking methods depending on the cultural sphere etc. It is not communicated correctly and, in the end, you may not know what food is until you actually eat it.

  If the explaining person can explain by explaining the difference or the similar cooking or cooking method in the listener's cultural sphere in consideration of the listener's cultural sphere etc., the listener is also easy to imagine. However, it is often the case that the person who explains is not knowledgeable about the listener's cultural sphere, and in such cases it is difficult to compare or explain the differences.

  This can also occur when speakers with different background knowledge have a conversation in the same language.

  2. Description of the Related Art Conventionally, a system is known in which supplementary information is always added to a translated word for a specific proper noun when translated. However, supplementary information is always output every time a specific specific expression appears during a voice conversation, which is rather troublesome and hinders a smooth conversation.

  Also, a system is known in which when a user interacts with an audio guide, the audio guide detects the user's confusion and provides information for resolving the confusion. However, this system does not function when the user does not feel confused, and the voice guide is not able to provide information in advance so as not to be confused by the user. Furthermore, this system is not intended for conversations between people with different background knowledge.

JP 2004-220416 A JP 2000-259177 A

  There has been no known technique for assisting a dialogue by presenting information for supplementing knowledge in accordance with the dialogue when the speaker speaks in different languages or in the same language.

  The present embodiment provides a dialogue support apparatus, method, and program capable of supporting a dialogue by presenting information for supplementing knowledge according to the dialogue when a speaker conducts a dialogue in different languages or in the same language. The purpose is to provide.

  According to the embodiment, an input unit, a voice recognition unit, a dialogue history database, an estimation unit, a determination unit, a generation unit, a selection unit, and a presentation unit are provided. The input unit inputs sound related to dialogues by a plurality of speakers. The voice recognition unit recognizes the input voice and converts it into corresponding text information. The dialogue history database stores all or part of the text information as a dialogue history. The estimation unit estimates an utterance action based on the text information. The determination unit determines whether to present supplementary information based on the estimated speech act. A generation part produces | generates the candidate of supplement information, when it determines with presenting the said supplement information. The selection unit uses the dialog history to select a candidate to be presented from the supplementary information candidates. The presenting unit presents the selected supplementary information.

It is a figure which shows the structural example of the dialogue assistance apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the process sequence of the dialogue assistance apparatus which concerns on 1st Embodiment. It is a figure for demonstrating the use situation of the dialog assistance apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the process sequence of a speech recognition process part. It is a flowchart which shows another example of the process sequence of a speech recognition process part. It is a flowchart which shows an example of the process sequence of a machine translation part. It is a flowchart which shows an example of the process sequence of a speech act estimation part. It is a figure which shows the example of a speech act tag. It is a flowchart which shows an example of the process sequence of a presentation necessity determination part. It is a figure which shows an example of a supplementary information presentation availability database. It is a flowchart which shows an example of the process sequence of a presentation candidate production | generation part. It is a figure which shows an example of a supplementary explanation database. It is a flowchart which shows an example of the process sequence of a candidate selection part. It is a figure for demonstrating the operation example of the dialogue assistance apparatus which concerns on 1st Embodiment. It is a figure which shows the example of a dialog. It is a figure which shows the other structural example of a dialog assistance apparatus. It is a figure which shows the further another structural example of a dialog assistance apparatus. The example of a supplement information presentation content database is shown. It is a figure which shows the example of a dialog. It is a figure which shows the structural example of the dialogue assistance apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the process sequence of the dialogue assistance apparatus which concerns on 2nd Embodiment.

  Hereinafter, a dialogue support apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that, in the following embodiments, the same numbered portions are assumed to perform the same operation, and repeated description is omitted.

(First embodiment)
If two speakers trying to interact cannot use a common language (natural language) (for example, they both use only their native language and cannot understand the other's native language) Or, one or both of these speakers can use multiple languages, but there is no language that they can use in common, for example, a translator between them By doing so, it is possible to support the speakers using different languages (for example, their native languages). However, there are often differences in background knowledge between speakers. Therefore, as long as the translator correctly translates the information emitted by one speaker for the other speaker, it is not necessarily expected that the information will be correctly transmitted from one speaker to the other. Sometimes. This can also occur when two speakers with different background knowledge have a conversation in a common language.

  In the first embodiment, a case will be described as an example where information for supplementing knowledge is presented according to a dialogue when machine translation is involved in the dialogue.

  In the first embodiment, two speakers using the dialogue support apparatus are referred to as a first speaker and a second speaker.

  In the following, the first speaker is the customer, the second speaker is the customer service (for example, the store clerk who is the customer), and the first language that the first speaker understands and speaks (for voice input) is Japan. A specific example is used in which the second language that the second speaker understands and speaks (for speech input) is English. Of course, the present embodiment is not limited to this, and the first language can be a language other than Japanese, and the second language can be a language other than English.

  In the following description, the first speaker is a restaurant customer and the second speaker is a restaurant customer. Of course, the present embodiment is not limited to this, and various customer communication between different languages such as communication between different languages when providing other services, communication between different languages when selling products, etc. It is applicable to.

  In the following description, a customer service support device that supports customer communication in which a customer service a customer is described as an example of a dialog support device that supports a dialog. However, the present embodiment can also be applied to conversations other than customer service. is there.

  In the following, when the first speaker and the second speaker are talking using different languages, it is determined that there is unknown information for one speaker and the translation result is output. Shows an example in which supplemental information (for example, explanatory text) is presented to the one speaker at a predetermined timing. Here, one speaker is described as the first speaker, but the present invention is not limited to this. In addition, it is possible to determine that there is unknown information and present the supplementary information for the second speaker when outputting the translation result, not both speakers but both speakers .

  First, an example of a usage situation of the customer service support device (dialog support device) of the present embodiment will be shown with reference to FIG.

  First, in STEP-C1, the second speaker (customer) speaks the content for listening to an order (for example, “Are you ready to order?”) In English. Then, in STEP-C2, the customer service support device translates the content into Japanese and presents (speaks and / or displays) the translated content (for example, “Is your order decided?”).

  In STEP-C3, the first speaker (customer) who heard and / or read the above-mentioned translation content asks for a recommended dish in Japanese (for example, “What is the recommended dish?” ). Then, in STEP-C4, the customer service support device translates the content into English and presents (speaks and / or displays) the translated content (for example, “Which one do you recommend?”).

  In STEP-C5, a second speaker who has heard and / or read the translated content speaks in English the content for answering a recommended dish (for example, “I recommend Wiener Schnizel.”). Then, in STEP-C6, the customer service support device translates the content into Japanese, and presents (speaks and / or displays) the translated content (for example, “Winner Schnitzel is recommended.”).

  Here, the customer service support apparatus according to the present embodiment further determines whether or not supplementary information needs to be presented as will be described in detail, and if it is determined that supplementary information needs to be presented, For example, in STEP-C7, supplementary information (in this example, in Japanese corresponding to the first speaker (customer), for example, “Winner Schnitzel is a Viennese cutlet and is a typical Austrian dish.” .)) Is presented (spoken and / or displayed).

  In FIG. 1, the function structural example of the customer service assistance apparatus of this embodiment is shown.

  As shown in FIG. 1, the customer service support apparatus according to the present embodiment includes an input unit 101, a speech recognition unit 102, a machine translation unit 103, a speech act estimation unit 104, a presentation necessity determination unit 105, a presentation candidate generation unit 106, A candidate selection unit 107 and a presentation unit 108 are included.

  The input unit 101 accepts a voice in the first language uttered by the first speaker and a voice in the second language uttered by the second speaker. For example, the input unit 101 may input a voice using a microphone, digitize the voice, and pass it to the voice recognition unit 102.

  The speech recognition unit 102 recognizes speech information in an input language that is a first language or a second language, and converts the speech information into text information (pre-translation text) in the input language.

  Based on the text information, the machine translation unit 103 generates a text (translation text) translated from the input language (first language or second language) into the other language (second language or first language). .

  The utterance action estimation unit 104 estimates the utterance action from the translated text and / or the pre-translation text.

  The dialogue history database 121 is a database for storing a dialogue history based on pre-translation text obtained by the speech recognition unit 102 and a dialogue history based on translation text obtained by the machine translation unit 103. The dialogue history database 121 may further store a dialogue history based on the speech act estimated by the speech act estimating unit 104.

  The dialogue history database 121 is referred to by the candidate selection unit 107 when selecting a candidate.

  The supplementary information presentation necessity database 122 is a database for storing information used for determining whether or not supplementary information is presented.

  The presentation necessity determination unit 105 determines whether there is unknown information for the first speaker and whether supplementary information for the unknown information is presented based on the estimated utterance action.

  The supplementary explanation database 123 is a database for storing knowledge information (supplementary explanation) used when generating supplementary information candidates (presentation candidates) to be presented.

  The presentation candidate generation unit 106 generates a candidate for supplementary information (or supplementary explanation) by referring to the supplementary explanation database 123 when the presentation necessity judgment unit 105 determines that presentation of supplementary information is necessary. (Or get).

  The candidate selection unit 107 selects what should be presented (or what should be used for presentation) from the generated supplementary information (or supplementary explanation) candidates. At that time, the candidate selection unit 107 refers to the dialogue history database 121 to make a selection.

  The presentation unit 108 presents the translation result and presents supplementary information at a predetermined timing. For example, the presentation unit 108 may present the translation result and supplementary information audibly (for example, uttered from a speaker), or may be presented visually (for example, a display such as a liquid crystal display). (Displayed on the screen). Note that the same presentation method may be used for the translation result and the supplemental information, or the supplementary information may be presented by a method different from the translation result. For example, both the translation result and the supplemental information may be presented audiovisually, the translation result may be displayed only auditorily, the supplemental information may be presented only visually, or any other presentation A method is also possible.

  In addition, when the translation unit visually displays the translation result, the presentation unit 108 may also visually display the pre-translation text.

  Furthermore, when presenting supplementary information, the presentation unit 108 may perform processing such as sounding, blinking the display, or vibrating the device in order to inform the user that supplementary information is being presented.

  In the present embodiment, the supplemental information is mainly described in the first language for the customer. However, the supplementary information is presented in the second language for the customer service, It may be possible to present in one language and in a second language. In these cases, even when supplementary information for customers is presented audibly, supplementary information for customers only You may make it present visually so that it may not be heard.

  FIG. 2 shows an example of the overall processing procedure of this embodiment.

  When the user has a conversation, the voice of any speaker enters the input unit 101 (step S1), the voice recognition by the voice recognition unit 102 (step S2), and the machine translation by the machine translation unit 103 (step S3) are sequentially performed. Is called. For example, the presentation unit 108 may present the machine translation result at an appropriate timing after step S3 (for example, immediately after step S3).

  Further, the utterance act estimation by the utterance act estimation unit 104 (step S4) and the supplemental information presentation necessity determination (step S4) by the presentation necessity determination unit 105 are sequentially performed.

  If it is determined that the presentation is necessary (step S5), the candidate candidate generation unit 106 generates a supplemental information candidate (presentation candidate) (step S6), and then the candidate selection unit 107 displays the presentation candidate. One or more presentation candidates to be used for presentation are selected from the list (step S7). In addition, a presentation candidate is good also as supplementary information as it is, and in step S7, the candidate selection part 107 produces | generates an explanatory text based on a presentation candidate, and is good also as supplementary information. Then, the presentation unit 108 presents supplemental information at an appropriate timing (step S8). And it returns to step S1 and repeats a series of processings.

  On the other hand, when it is determined that the presentation is unnecessary (step S5), steps S6 to S8 are skipped, the process returns to step S1, and a series of processes is repeated.

  Hereinafter, processing examples of each configuration will be described in more detail with reference to FIGS.

(Voice recognition unit 102)
FIG. 4 shows an example of processing of the voice recognition unit 102.

  In step S <b> 11, the voice received by the input unit 1 is input from the input unit 1.

  In step S12, the input speech is converted into text by performing speech recognition (pre-translation text is generated).

  In step S13, the pre-translation text is output.

  If language determination is required during speech recognition, as shown in FIG. 5, the speech recognition unit 102 performs language determination in step S14. In the language determination, the language may be directly specified from the input voice. Instead, for example, the correspondence between the customer (store clerk) and the characteristics of the voice and the language to be used is set in advance. Alternatively, it may be determined whether or not the language corresponds to the customer by determining whether or not the speaker is a customer. Alternatively, features extracted from the utterance content may be used. Alternatively, the user may instruct language selection with a button or the like simultaneously with the utterance or before or after the utterance. Alternatively, in the input unit 101, a dedicated microphone is prepared for each speaker, the correspondence between the microphone and the language is set, and the speaker and the language are specified depending on which microphone the voice is input from. Is also possible. Alternatively, the direction in which the utterance is spoken is associated with the speaker, the direction in which the utterance is made is estimated, and the speaker may be specified based on the estimation result. Alternatively, buttons for the first speaker and the second speaker may be prepared, and the speaker may select the button by pressing the button. At that time, for example, when the first speaker / second speaker is selected and the voice is input, the second speaker / first speaker is automatically selected to be alternately selected. It is also possible to eliminate the need for the button operation when uttering a voice and press the button of the speaker only when the same speaker continuously utters. In addition to these, various methods for specifying or specifying a language are possible. In this case, in step S12, speech recognition and text conversion are performed according to the result of language determination in step S14.

(Machine translation unit 103)
FIG. 6 shows an example of processing of the machine translation unit 103.

  In step S21, the pre-translation text converted into text by the speech recognition unit 102 is input.

  In step S22, the language of the input text is determined.

  When language determination is not performed in the speech recognition unit 102, the language determination may be the same as the language determination in the speech recognition unit 102 described above. Instead, for example, language determination may be performed from the pre-translation text.

  When language determination is performed in the speech recognition unit 102, the language determination may be performed by using the language determination result by the speech recognition unit 102, omitting step S22, or performing language determination independently in step S22. Also good.

  If the input language is the first language, translation from the first language to the second language is performed in step S23, and text (translation text) in the second language is output in step S24.

  On the other hand, if the input language is the second language, translation from the second language to the first language is performed in step S23, and text (translation text) in the first language is output in step S24.

  For example, in step S22, it is determined whether the input language is Japanese or English. If it is determined that the input language is English, the input text is translated from English to Japanese and the Japanese translation text is output. .

(Speech act estimation unit 104)
FIG. 7 shows an example of processing of the speech act estimating unit 104.

  The speech act estimation unit 104 may always be able to determine both languages using the translated text in the speech act estimation process (step S32), or the language of step S22 of the machine translation unit 103. Using the determination result or the determination result of the language determination in step S14 of the speech recognition unit 102, it is possible to select before translation or after translation and always infer the same language.

  Here, a case where text is always input in the second language is illustrated.

  In step S31, text in the second language before translation or after translation (that is, translation text relating to the first speaker or text before translation relating to the second speaker) is input.

  In step S32, the speech act tag is estimated using the text obtained in step 402.

  Note that the previous utterance may be taken into account using the dialogue history when estimating the utterance action tag. That is, in step S31, history information (utterance content and speaker tag) in the dialogue history database 121 is also input, and in step S32, the text obtained in step 402 and the history information (utterance content and speaker) are input. Tag) may be used to estimate the speech act tag.

  In step S33, the speech act tag obtained as the speech act tag estimation result is output.

  FIG. 8 shows an example of the speech act tag. In this example, speech act tags such as “greeting”, “acknowledgment / apology”, “conformity”, “response”, “question”, “suggestion”, “information transmission”, and “request” are defined.

  Various methods such as a method using a statistical method such as machine learning or a method using a rule from a clue expression can be used for estimating the speech act. For example, if the input is "How about Wiener Schnitzel?", If the input is "Suggesting action tag =" Suggestion "," Yes ", then the speaking action tag =" Response ", If the input is “1”, the utterance action tag = “request”, and “What kind of dish is Wiener Schnitchel?”, The utterance action tag = “question”. Can be estimated.

(Presentation necessity determination unit 105)
FIG. 9 shows an example of processing of the presentation necessity determination unit 105.

  In step S41, the speech act tag output by the speech act estimating unit 104 and the determination result of the language determination in step S22 of the machine translation unit 103 or the determination result of the language determination in step S14 of the speech recognition unit 102 are used. Perform speaker determination.

  For example, if the first speaker is a customer and the second speaker is a customer, if the customer language is set to English as the second language, the language determination result is used. If the language is the second language, the customer who is the second speaker is obtained as the speaker tag.

  Instead of this, for example, the correspondence between the customer service (clerk) and the characteristics of the voice may be set in advance, and it may be determined from the characteristics of the input voice whether or not the speaker is a customer service. .

  In addition, as described above, when the speech recognition unit 102 or the machine translation unit 103 performs speaker determination for language determination, the result may be used.

  Next, in step S42, an utterance action tag and a speaker tag are input, and in step S43, the supplementary information presentation necessity database 122 is referred to using the input combination of the utterance action tag and the speaker tag.

  FIG. 10 shows an example of the supplementary information presentation necessity database 122. In this example, a plurality of triple data sets of “speaker tag”, “utterance action tag”, and “presentation necessity” are defined, and “speaker tag” and “speech action tag” corresponding to a combination of “ The necessity determination can be made by referring to the contents of the “necessity of presentation” column. In this example, for the sake of convenience, “presentation is required” is indicated by ○, and “not required” is indicated by ×.

  Furthermore, the presentation timing for presenting supplementary information may be described together with each condition. When describing the presentation timing, supplementary information will be presented according to the contents of the “presentation timing” column. When not presenting the presentation timing, the timing is set at a predetermined timing (for example, immediately or when serving customers). Supplementary information will be presented).

  In the present embodiment, the supplemental information is described by way of example in the first language used by the first speaker. However, the language used when presenting the supplemental information for each condition is described. May be provided. In this case, supplementary information is presented in the language described in the “Language” column. Thereby, for example, it can be instructed to present the supplementary information in the first language for the customer and to present the supplementary information in the second language for the customer service. In this case, the “language” field may describe the first language or the second language. Furthermore, it may be possible to describe both the first language and the second language in the “Language” column.

  In the case of the example of FIG. 10, in step S43, the content of the “presentation necessity” column corresponding to the combination of the “utterance action tag” and the “speaker tag” input in step S42 (when the “presentation timing” column is provided) In the “presentation necessity” column and the “presentation timing” column).

  In step S44, the content of the “presentation necessity” column corresponding to the combination of the “utterance action tag” and the “speaker tag” input in step S42 on the supplementary information presentation necessity database 122 is “presentation required”. Is determined.

  If it is determined that “presentation is necessary”, a presentation request is issued in step S45, and if it is determined that presentation is not required, a non-presentation request is issued in step S46.

  When the “presentation timing” field is provided, the content of the “presentation timing” field may be transmitted to the presentation unit 108 via the presentation candidate generation unit 106 and the candidate selection unit 107, for example, by adding to the presentation request. Alternatively, the content of the “presentation timing” field may be directly communicated to the presentation unit 108 (or the candidate selection unit 107 when the candidate selection unit 107 uses the content of the “presentation timing” field). .

  When the “presentation timing” column is provided, the contents of the “presentation timing” column may be acquired before step S46 only when it is determined in step S44 that the “presentation is required”. good.

(Presentation candidate generation unit 106)
FIG. 11 shows an example of processing of the presentation candidate generation unit 106.

  Only when the presentation request is output by the presentation necessity determination unit 105, the processing of the presentation candidate generation unit 106 is performed.

  The presentation candidate generation unit 106 may always be able to determine both languages using the translated text in the presentation candidate generation process (step S52), or the language in step S22 of the machine translation unit 103. By using the determination result or the determination result of the language determination in step S14 of the voice recognition unit 102, it is possible to select before translation or after translation and always be able to determine in the same language. Here, a case where text is always input in the second language is illustrated.

  In step S51, text in the second language before translation or after translation (that is, translation text relating to the first speaker or text before translation relating to the second speaker) is input.

  Next, in step S52, keywords are extracted from the text.

  In step S53, it is confirmed whether or not each keyword extracted in step S52 exists on the supplementary explanation database 123. If a supplementary explanation of one or more keywords is registered in the supplementary explanation database 123 for a certain keyword, the supplementary explanation is output in step S54. On the other hand, if no keyword is registered for a certain keyword, null is output in step S55.

  FIG. 12 shows an example of the supplementary explanation database 123. The supplementary explanation database 123 includes a “word” field that is an indexed keyword, a “language” field that indicates language information related to the supplementary explanation, a “priority” field that indicates a priority among a plurality of supplementary explanations, and a language for each language. There is an item in the “supplemental explanation” column given correspondingly. In this specific example, “Wiener Schnitzel” in English, which is the second language used by the customer, is registered as a keyword as unknown information for customers using Japanese or Italian. In the example of FIG. 12, the lower the numerical value described in the “priority” column, the higher the priority.

  For example, in the specific example of FIG. 12, when the text input in step S51 is always in the second language, when “Wiener Schnitzel” is obtained as a keyword in step S52, in step S54, “Vienna cutlet” “oiled” Output three supplementary explanations: “Fried beef” and “Austrian typical dishes”.

(Candidate selection unit 107)
FIG. 13 shows an example of processing of the candidate selection unit 107.

  In step S61, the supplementary explanation output by the presentation candidate generation unit 106 is input as a presentation candidate.

  In step S62, one or more supplement information to be presented is selected from the plurality of supplement information.

  When selecting a candidate, the information in the dialogue history database 121 (for example, only the utterance content or the utterance content and the speaker tag) is examined, and a supplementary explanation excluding the supplementary explanation similar to the already uttered content is selected. It is desirable to do. Alternatively, the known information may be determined based on the combination of the speaker tag and the utterance action tag from the information in the dialogue history database 121, and the supplementary explanation excluding the supplementary explanation similar to the known information may be selected. For example, in a dialogue history, if the speaker tag = "customer", "I want to eat Wiener Schnell, but ..." and the speech act tag = "wish" , Because we should know about Wiener Schnitchell, do not select supplementary explanations based on general knowledge such as “Representative dish of Austria” of JP-3 (Japanese, priority 3) in FIG. Such a judgment may be made.

  In this selection, for example, a predetermined number (for example, one, two, etc.) of supplementary explanations may be selected in order from the highest priority.

  In this selection, the content of the “presentation timing” column may be taken into consideration. For example, if the presentation timing is “immediately after determination”, the supplementary explanation with the highest priority may be selected. Alternatively, for example, if the presentation timing is “immediately after the next utterance”, a supplementary explanation excluding a supplementary explanation similar to the utterance content used in the next utterance may be selected.

  In step S63, supplemental information (herein, an explanatory text) is generated from the selected one or more supplementary explanations.

  In step S64, the generated explanatory text is output.

  For example, when a customer asks “What kind of dish is Wiener Schnitchel?”, The speaker tag = “customer” and the utterance action tag = “question”, so “after the customer's answer” It is scheduled to present supplementary explanations or explanations generated based on the supplementary explanations. Therefore, when the customer replied, “Wiener Schnitchell is beef raised with oil,” the supplementary explanation of JP-2 (Japanese, priority 2) in the supplementary explanation database 123 was answered. Excluding this from the supplementary explanation, for example, from the supplementary explanation of JP-2 (Japanese, priority 2) and the supplementary explanation of JP-3 (Japanese, priority 3), "Vienna cutlet, Austrian A description such as “Typical dish” may be generated and output.

  Also, for example, when the customer proposes “How about Winner Schnitzel?”, Since the speaker tag = “customer” and the utterance action tag = “suggest”, “just after the customer speaks” ", Supplementary information (descriptive text) will be presented. At this time, using two supplementary explanations (JP-1 and JP-2) with high priority, a supplementary explanation such as “Vietnamese cutlets will be beef raised with oil” is generated and output. You may make it do.

  Alternatively, when the speaker tag = “customer” and the utterance action tag = “suggestion”, the utterance action tag and the supplementary explanation database 123 are set so as not to give a specific cooking method. A priority for each customer tag may be attached.

  Note that the selected supplementary explanation may be used as supplementary information as it is.

(Presentation unit 108)
As described above, the presentation unit 108 presents at least supplemental information at a predetermined timing.

  FIG. 14 shows an operation example of the customer service support apparatus of this embodiment.

  When a customer asks "What is Wiener Schnitchell" in Japanese, the translation result in English is obtained and presented.

  Here, since the speaker tag = “customer” and the utterance action tag = “question”, the supplementary information is presented at the timing “after the customer's answer”. As the supplementary explanation, for example, the supplementary explanation “Vienna cutlet” of JP-1 (Japanese, priority 1) is selected.

  When the customer responds to the above question, “Beef raised with oil”, the translation result in Japanese is obtained and presented. At this timing, an explanatory note such as “Vienna cutlet” is presented as supplementary information.

  FIG. 15 shows an example of dialogue using the customer service support apparatus of this embodiment and an example of supplementary explanation.

  In the dialogue example 1, if the customer asks “What is the content of the recommended course today?”, Since the speaker tag = “customer” and the speech act tag = “question”, the presentation request is made. Although there are no keywords registered in the supplementary explanation database, no supplementary information is presented.

  Next, if the customer responds “It is XXXXX, Wiener Schnitchell, XXXXX”, the speaker tag = “customer” and the utterance action tag = “answer”. A presentation request is issued. The presentation timing is, for example, “immediately after the customer service utterance”. And there is a keyword registered in the supplementary explanation database = Wiener Schnitzel, and now it is “immediately after the customer's utterance”, so at this time, for example, “Wiener Schnitzel is a Viennese cutlet. "Is presented as supplementary information.

  In Dialogue Example 2, if the customer asks "What is Wiener Schnell?", The presentation request is issued because the speaker tag = “customer” and the speech act tag = “question”. . The presentation timing is, for example, “after the customer's answer”. Then, there is a keyword = Wiener Schnitchle registered in the supplementary explanation database, and the explanatory text is presented as supplementary information.

  Here, suppose that the customer responds, "Our Schnitzel is fried beef." In this case, the content similar to JP-2 (Japanese, priority 2) supplementary explanation “beef raised with oil” was spoken, so this was excluded from the candidates, for example, JP-1 (Japanese, The supplementary explanation “Vienna style cutlets” of priority 1) is selected. Since it is “after the customer's answer”, an explanatory sentence such as “so-called Viennese cutlet” is presented as supplementary information at this timing.

  In the above description, the combination of {first speaker = customer = first language = Japanese, second speaker = customer = second language = English} has been described as an example, but the present invention is not limited to this. Any other combination is possible. For example, {first speaker = customer = first language = English, second speaker = customer = second language = Japanese}, for example, a combination of Japanese and Chinese, or A combination of English and Chinese may be used.

  In addition, the combination of languages may be fixed in advance to two languages, or the user may freely select from three or more languages. Further, one or both languages of the customer and the customer service may be fixed in advance to one language, or the user may freely select.

  The first speaker may be a customer service and the second speaker may be a customer. Further, the two speakers need not have the customer-customer relationship.

  In addition, in the above, the first speaker is the target to present supplementary information after judging that there is unknown information, but it is also possible to be the second speaker or both speakers . In addition, whether to target the first speaker, the second speaker, or both speakers may be fixed in advance, and the user can freely select. May be.

  As described above, according to the present embodiment, when a speaker interacts in different languages or in the same language, it is possible to support the conversation by presenting information for supplementing knowledge according to the conversation. Become.

(First modification)
FIG. 16 shows a configuration example when language selection is performed using buttons. In this case, in the configuration described so far, the language selection input unit 111 is provided in front of the input unit 101, and the language input by the language selection input unit 111 is designated or selected.

(Second modification)
FIG. 17 shows a configuration example in the case of determining the presentation content. In this case, a presentation content determination unit 112 is provided instead of the presentation necessity determination unit 105. In addition to the supplementary explanation database 123, a predetermined number of other databases are provided. Here, as an example, it is assumed that a product information database 125 that holds recommended information about each dish, for example, and a management information database 126 that holds management information such as the remaining number of each dish, for example.

  Further, the supplement information presentation content database 124 is used instead of the supplement information presentation necessity database 122.

  FIG. 18 shows an example of the supplementary information presentation content database 124. The supplementary information presentation content database 124 is different from the supplementary information presentation necessity database 122 in that a “reference database” column is provided instead of the “display necessity” column. In the “reference database”, not the necessity of presentation, but a database to be referred to is described. For example, DB1 indicates the supplementary explanation database 123, DB2 indicates the product information database 125, and DB3 indicates the management information database 126. Note that any database may be described in the “reference database” field, and the “reference database” field is left blank (that is, the blank field indicates that presentation is unnecessary), It may be possible.

  In the present embodiment, the presentation content determination unit 112 identifies a database to be referred to based on the content of the “reference database” column. When it is possible to leave the “reference database” field blank, the presentation content determination unit 112 identifies the necessity of presentation and the database to be referred to when presentation is necessary.

  FIG. 19 shows an example of interaction in the present embodiment. In the case of a customer question or a customer service suggestion, the point that supplementary information is presented is the same as the above-described system. (A) Customer's question “What is Wiener Schnitchell?” And (b) Customer service suggestion “How about Winner Schnitzel?” Refer to supplementary explanation database 123 In both cases, the same result as in the first embodiment is obtained.

  On the other hand, when the customer requests “Please winner schnitzel” as shown in (c), the management information database 126 is referred to. For example, the management information database 126 holds the remaining number for each dish, searches for the remaining number corresponding to the Wiener Schnitchell, and presents supplementary information such as “one serving”, You may not accept orders for dishes that have already been sold out. At this time, when the second language is designated as “language” in the supplementary information presentation content database 124, for example, supplementary information such as “remaining one serving” is presented to the customer in the second language. Also good.

  In addition, when the customer confirms “It is Wiener Schnitchel” as in (d), the product information database 125 is referred to, for example, for the customer and / or the customer. Further, supplementary information such as recommended information (for example, information such as a recommended wine list suitable for Wiener Schnitchell) may be presented.

  Note that the various databases described here are examples, and supplementary information can be generated and presented using various databases.

  It should be noted that one or both of the first modification and the second modification in the first embodiment can be implemented in combination with the second embodiment.

(Second Embodiment)
In the following, the second embodiment will be described with a focus on differences from the first embodiment.

  In the first embodiment, supplementary explanation is presented when machine translation is used for dialogue between speakers using different languages. However, in the second embodiment, speakers with different background knowledge are presented. When a dialogue is conducted in the same language without using machine translation, a supplementary explanation is presented.

  FIG. 20 shows a functional configuration example of the customer service support apparatus of the present embodiment.

  As shown in FIG. 20, the customer service support apparatus according to the present embodiment includes an input unit 101, a speech recognition unit 102, a speaker identification unit 1103, a speech act estimation unit 104, a presentation necessity determination unit 105, and a presentation candidate generation unit 106. The candidate selection unit 107 and the presentation unit 108 are included. That is, in the configuration example of FIG. 1, the speaker specifying unit 1103 is provided instead of the machine translation unit 103.

  The input unit 101 receives the voice of the first speaker and the voice of the second speaker in a specific language (for example, a language common to both speakers such as Japanese and English).

  The voice recognition unit 102 recognizes voice information in a specific language and converts it into text information.

  The speaker specifying unit 1103 specifies a speaker.

  The speaker specifying unit 1103 may use the speaker specifying method (except for specifying the speaker by language) exemplified in the first embodiment, or any other method.

  Note that when the voice recognition unit 102 has a function of specifying a speaker, the speaker specifying function of the voice recognition unit 102 may be the speaker specifying unit 1103.

  The utterance action estimation unit 104 estimates the utterance action from the text information.

  The dialogue history database 121 is the same as that of the first embodiment except for the following points. In this embodiment, since both speakers use a common language and do not perform machine translation, the conversation history by the translated text obtained by the machine translation unit is not saved.

  The supplementary information presentation necessity database 122 is the same as in the first embodiment.

  The presentation necessity determination unit 105 is the same as that in the first embodiment. In this embodiment, the speaker specifying unit 1103 can use the speaker specifying result.

  The supplementary explanation database 123 is the same as that of the first embodiment except for the following points. In this embodiment, since both speakers use a common language and do not perform machine translation, the “language” field is unnecessary, and the same language is used in the “word” field and the “supplemental explanation” field. The

  The presentation candidate generation unit 106 is the same as that in the first embodiment.

  The candidate selection unit 107 is the same as that in the first embodiment.

  The presentation unit 108 is the same as that in the first embodiment. However, in this embodiment, the translation text is not presented. Further, the presentation unit 108 may present only supplemental information.

  FIG. 21 shows an example of the overall processing procedure of this embodiment.

  When the user has a conversation, the voice of any speaker enters the input unit 101 (step S101), voice recognition by the voice recognition unit 102 (step S102), and speaker identification by the speaker identification unit 1103 (step S103). Are performed sequentially.

  Further, the utterance act estimation by the utterance act estimation unit 104 (step S104) and the supplemental information presentation necessity judgment (step S104) by the presentation necessity judgment unit 105 are sequentially performed.

  When it is determined that the presentation is necessary (step S105), the candidate for supplementary information (presentation candidate) is generated (step S106) by the presentation candidate generation unit 106, and then the candidate for presentation is displayed by the candidate selection unit 107. One or more presentation candidates to be used for presentation are selected from the list (step S107). In addition, a presentation candidate is good also as supplementary information as it is, and in step S107, the candidate selection part 107 produces | generates an explanatory note based on a presentation candidate, and is good also as supplementary information. Then, the presentation unit 108 presents supplementary information at an appropriate timing (step S108). And it returns to step S101 and repeats a series of processes.

  On the other hand, when it is determined that the presentation is unnecessary (step S105), steps S106 to S108 are skipped, the process returns to step S101, and a series of processes is repeated.

  The configuration described with reference to FIGS. 20 and 21 may be further implemented by combining one or both of the first modification and the second modification described in the first embodiment.

  In addition, when applying a 2nd modification, the supplementary information presentation content database 124 may be the same as that of 1st Embodiment.

  Note that the database described in each of the above-described embodiments or modifications may exist, for example, inside the dialogue support apparatus. Alternatively, a part or all of the database may exist on a network such as a LAN, and the dialogue support apparatus may acquire information from the database via the network.

The instructions shown in the processing procedure shown in the above embodiment can be executed based on a program that is software. The general-purpose computer system stores this program in advance and reads this program, so that it is possible to obtain the same effect as that obtained by the dialogue support apparatus of the above-described embodiment. The instructions described in the above-described embodiments are, as programs that can be executed by a computer, magnetic disks (flexible disks, hard disks, etc.), optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD). ± R, DVD ± RW, etc.), semiconductor memory, or a similar recording medium. As long as the recording medium is readable by the computer or the embedded system, the storage format may be any form. If the computer reads the program from the recording medium and causes the CPU to execute instructions described in the program based on the program, the same operation as the dialogue support apparatus of the above-described embodiment can be realized. Of course, when the computer acquires or reads the program, it may be acquired or read through a network.
In addition, the OS (operating system), database management software, MW (middleware) such as a network, etc. running on the computer based on the instructions of the program installed in the computer or embedded system from the recording medium implement this embodiment. A part of each process for performing may be executed.
Furthermore, the recording medium in the present embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.
Further, the number of recording media is not limited to one, and when the processing in this embodiment is executed from a plurality of media, it is included in the recording medium in this embodiment, and the configuration of the media may be any configuration.

The computer or the embedded system in the present embodiment is for executing each process in the present embodiment based on a program stored in a recording medium. The computer or the embedded system includes a single device such as a personal computer or a microcomputer. The system may be any configuration such as a system connected to the network.
In addition, the computer in this embodiment is not limited to a personal computer, but includes an arithmetic processing device, a microcomputer, and the like included in an information processing device, and is a generic term for devices and devices that can realize the functions in this embodiment by a program. ing.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 101 ... Input part, 102 ... Speech recognition part, 103 ... Machine translation part, 104 ... Speech action estimation part, 105 ... Presentation necessity judgment part, 106 ... Presentation candidate production | generation part, 107 ... Candidate selection part, 108 ... Presentation part, DESCRIPTION OF SYMBOLS 111 ... Language selection input part, 112 ... Presentation content determination part, 1103 ... Speaker specific part, 121 ... Dialog history database, 122 ... Supplementary information presentation necessity database, 123 ... Supplementary explanation database, 124 ... Supplementary information presentation content database, 125... Management information database, product information database 126.

Claims (14)

A generating unit that generates supplementary information from a supplementary explanation database based on a speech act tag and a speaker tag based on input text information;
And an output unit that outputs the supplementary information. A dialogue history database for storing all or part of the text information as a dialogue history;
An estimation unit for estimating a speech act tag based on the text information;
A determination unit that determines whether to output the supplementary information based on the estimated speech act tag and the speaker tag;
A selection unit that selects one or more presentation candidates to be used for presentation among the supplementary information, and
The interactive apparatus according to claim 1, wherein the generation unit generates the supplemental information when it is determined to output the supplemental information. A speaker specifying unit for specifying a speaker for the text information;
The dialogue apparatus according to claim 2, wherein the determination unit determines whether to output the supplementary information based on the estimated speech act tag and the specified speaker. A machine translation unit that translates the text information in the first language and generates translated text information in a second language different from the first language;
The dialogue apparatus according to claim 2 or 3, wherein the dialogue history database stores all or part of text information before or after translation in the first language or the second language. .   Whether the determination unit specifies a speaker for the text information or acquires a speaker identification result, and outputs the supplementary information based on the estimated speech act tag and the identified speaker 5. The interactive apparatus according to claim 2, wherein the interactive device is determined.   6. The selection unit according to claim 2, wherein the selection unit excludes one or more presentation candidates whose corresponding contents are already included in the dialogue history from the candidates. The interactive apparatus according to claim 1.   The generating unit generates the supplemental information when a keyword registered in advance as unknown information exists in the text information that is determined to output the supplemental information. The interactive apparatus according to any one of claims 2 to 6. The generation unit executes generation with reference to a reference database,
8. The method according to claim 2, wherein the determination unit determines whether to refer to any one of the plurality of predetermined reference databases. 9. The interactive device described in 1.   9. The dialogue apparatus according to claim 2, wherein the estimation unit estimates the utterance action tag also using the dialogue history.   10. The dialog history database stores the estimated utterance action tag together with the text information that is the basis of the estimated utterance action tag. The interactive apparatus according to claim 1.   The dialogue apparatus according to claim 10, wherein the selection unit selects one to be presented from the one or more presentation candidates using the dialogue history including the utterance action tag. The determination unit also determines an output timing for outputting the supplemental information when it is determined to output the supplemental information,
The interactive device according to any one of claims 2 to 11, wherein the output unit outputs the supplementary information according to the output timing. A dialogue method of a dialogue device including a generation unit and an output unit,
The generating unit generating supplementary information from a supplementary explanation database based on a speech act tag and a speaker tag based on input text information;
The output unit includes a step of outputting the supplementary information. A program for causing a computer to function as an interactive device having a generation unit and an output unit,
A generating unit that generates supplementary information from a supplementary explanation database based on a speech act tag and a speaker tag based on input text information;
A program for causing a computer to realize an output unit that outputs the supplementary information.

Images