JP2009047920A - Device and method for interacting with user by speech - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech interaction device capable of easily correcting an error part without interrupting interaction. <P>SOLUTION: The speech interaction device comprises: a candidate generation section 112 which recognizes speech, and which generates a candidate of response and a likelihood for showing probability of the candidate of response; a response sentence generation section 113 for generating a response sentence including a phrase for expressing a content that the candidate of the most likely response is selected; an output section 102 for outputting synthesis speech of response sentence; a correction phrase generation section 114 for generating at least one correction phrase corresponding to the phrase included in the response sentence by analyzing the recognition result for the speech a user utters during an output of synthesis speech; a selection section 115 which obtains the candidate of the response including the phrase of the same meaning content with the generated correction phrase from the generated candidate of the response, and which selects the candidate of the most likely response in the obtained response candidates; and an update section 116 for updating the response sentence with the phrase expressing the content of the candidate of the selected response. The output section 102 outputs synthesis speech of the response sentence after updating. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an apparatus and a method for interacting with a user by executing an operation according to an input voice.

  In recent years, research on elemental technologies such as speech recognition, speech synthesis, and dialogue understanding has progressed, and by combining these, speech dialogue that can operate machines by uttering natural language speech without complicated button operations and command inputs Interfaces are being put into practical use.

  In addition, with the improvement in performance of digital home appliances and car navigation systems, it is becoming possible to implement a voice interaction interface that requires higher processing performance than the conventional user interface.

  However, many technical problems still remain in each of the above elemental technologies, and it is possible to always interpret the user's input voice to the system correctly and execute an operation that satisfies the user's request or output a response. Therefore, it is very difficult to realize a highly accurate system.

  For example, in order to interpret a user's request intention from speech, it is necessary to first extract language information from the speech waveform by speech recognition processing. However, even with this speech recognition processing, a correct result is not always obtained. For example, in a noisy environment, there is a problem that recognition accuracy is significantly reduced.

  Further, it is necessary to extract morpheme information and syntax information from the recognized linguistic information (text) and to further analyze the speech intention, but there is a possibility that an error occurs in any process. In particular, in order to understand dialogues such as extracting utterance intentions, very advanced language processing in consideration of context and the like is required. For this reason, it is very difficult for a speech dialogue processing system that can input a user's free utterance to always correctly interpret the user's utterance and avoid the occurrence of ambiguity.

  Therefore, along with improvements in elemental technology at each processing stage, measures are taken to enable the user to correct ambiguities and errors in the interpretation of the system using a human interface (HI).

  However, depending on the method of feedback of the system interpretation result to the user, the procedure may be complicated, or a series of correction processes of user input-system interpretation result response-user correction input-system interpretation correction-system operation execution Time may increase, which may stress the user.

  For example, when there are a plurality of interpretation candidates for the user's utterance, consider a method in which each interpretation candidate is fed back to the user by voice and the user selects the correct interpretation candidate. In this method, since interpretation candidates cannot be displayed as a list by text, it is necessary to output read-out speech corresponding to each interpretation candidate in order. For this reason, it takes time to output, and the processing load for the user to listen to and confirm the sound one by one increases.

  As a method for avoiding this, for example, a system in which the system outputs only the first interpretation candidate and accepts a correction input from the user can be considered. However, there is a problem that the correction process becomes complicated in the method of correcting simply by the procedure of response output-correction input-confirmation response output.

  In addition, a text display interface is also conceivable in which a list is displayed as text instead of being fed back by voice. However, when the display unit is small, an operation such as scrolling is required, and thus there may be a problem that the correction process becomes complicated as described above.

  As described above, in the voice interactive HI, a device is required so that the conversation between the person (user) and the machine can be smoothly advanced.

  For example, in Patent Document 1, a phrase in which a recognition error has occurred is automatically detected in the process of recognizing a speech from a user, and only the detected portion is presented to a source language speaker by text or speech. There has been proposed a technique for realizing an interactive interface that can be corrected smoothly by making corrections. In this method, since only the erroneous phrase is presented to the speaker, it is not necessary to confirm or re-enter the entire sentence, and the time required for correction can be shortened.

JP 2000-29492 A

  However, the method disclosed in Patent Document 1 has a problem in that, in the same way that erroneous recognition may occur in speech recognition, an error may also occur in the identification of a speech recognition error location. It was. There is also a problem that phrases other than the specified erroneous phrase cannot be corrected.

  In order to solve such a problem and realize a smooth dialogue, it is desirable to check not only the error location but also the entire interpretation result by voice and correct it by voice. However, even in this case, the general confirmation / correction method in which the corrected speech is accepted after the entire speech of the interpretation result is output may cause a problem that the progress of the dialogue is hindered.

  The present invention has been made in view of the above, and it is an object of the present invention to provide an apparatus and a method that can easily correct an error location without hindering dialogue.

  In order to solve the above-described problems and achieve the object, the present invention recognizes an input speech and generates a plurality of recognition result candidates, and a plurality of first recognition result candidates for the first speech. A candidate generation unit that analyzes and generates a response candidate corresponding to each of the plurality of first recognition result candidates and a likelihood that represents a likelihood of a response candidate for the first recognition result candidate; and the likelihood A response to the first candidate of the first recognition result including a phrase representing a candidate of a response to the first candidate of the first recognition result is selected. A response sentence generation unit that generates a sentence; an output unit that outputs a synthesized voice obtained by converting a response sentence to the first candidate of the first recognition result into a voice signal; and a second voice inputted during the output of the synthesized voice The candidate generated by the candidate generator Analyzing a candidate of the second recognition result for the speech and generating a corrected phrase that corrects the phrase included in the response sentence to the first candidate of the first recognition result; and a plurality of first recognition results A candidate for a response to another candidate of the first recognition result including the same phrase as the modified word is obtained from a candidate for the response to the candidate, and the likelihood among the candidates for the response to another candidate of the first recognition result is A selection unit that selects a response candidate for another candidate of the maximum first recognition result; and an update unit that updates the response sentence with a word of a response candidate for another candidate of the selected first recognition result. The output unit outputs the synthesized speech of the updated response sentence instead of the synthesized speech of the response sentence before the update when the response sentence is updated.

  The present invention is also a method capable of executing the above apparatus.

  According to the present invention, there is an effect that an error part can be easily corrected without obstructing the dialogue.

  Exemplary embodiments of an apparatus and a method according to the present invention will be described below in detail with reference to the accompanying drawings.

  The voice interactive apparatus according to the present embodiment interprets a user's input voice, outputs a response sentence corresponding to the interpretation result as a voice, and corrects a corrected voice for correcting the response sentence input during the output of the response sentence Is used to simultaneously update the interpretation result and response text, and output the updated response text.

  In the following, an example will be described in which a voice interactive apparatus is realized as a video recording / reproducing apparatus capable of recording / reproducing recorded broadcast programs, such as a hard disk recorder and a multimedia personal computer. The applicable apparatus is not limited to the video recording / reproducing apparatus, and can be applied to any apparatus that outputs a response corresponding to the user's input voice.

  FIG. 1 is a block diagram showing a configuration of a video recording / playback apparatus 100 according to the present embodiment. As shown in FIG. 1, the video recording / reproducing apparatus 100 mainly includes a microphone 131, a speaker 132, and a storage unit 120 as a hardware configuration. The video recording / playback apparatus 100 mainly includes a reception unit 101, a dialogue processing unit 110, an output unit 102, and a recording / playback unit 103 as software configurations.

  The microphone 131 is used to input voice spoken by the user. In addition, the speaker 132 converts a digital audio signal such as a synthesized voice obtained by synthesizing the response into an analog audio signal (DA conversion) and outputs the analog audio signal.

  The storage unit 120 records various data (details will be described later) such as action candidate groups, action fragments, and response phrase lists generated by the dialogue processing unit 110. The storage unit 120 can be configured by any generally used storage medium such as an HDD (Hard Disk Drive), an optical disk, a memory card, and a RAM (Random Access Memory).

  The accepting unit 101 performs sampling on the analog signal of the audio input from the microphone 131, converts it into a digital signal such as a PCM (pulse digital code modulation) format, and outputs it. Conventionally used A / D conversion technology or the like can be applied to the processing of the receiving unit 101.

  The dialogue processing unit 110 executes dialogue processing with the user by generating and outputting a response corresponding to the voice input from the user and a response sentence representing the content of the response. Specifically, the dialogue processing unit 110 first interprets a user request by voice recognition of a digital signal. Next, the dialogue processing unit 110 generates response candidates according to the interpretation result. Furthermore, the dialogue processing unit 110 generates a response sentence corresponding to the most likely candidate.

  The detailed function and configuration of the dialogue processing unit 110 will be described below. As shown in FIG. 1, the dialogue processing unit 110 includes a recognition unit 111, a candidate generation unit 112, a response sentence generation unit 113, a corrected phrase generation unit 114, a selection unit 115, and an update unit 116. ing.

  The recognizing unit 111 performs speech recognition on the audio digital signal output from the receiving unit 101 to generate a recognition result candidate representing a user request. Specifically, the recognition unit 111 performs speech recognition on the input digital signal and generates a recognition candidate group including at least one recognition candidate text. In the speech recognition processing by the recognition unit 111, all commonly used speech recognition methods using LPC analysis, Hidden Markov Model (HMM), dynamic programming, neural network, N-gram language model, and the like are used. Can be applied.

  FIG. 2 is an explanatory diagram illustrating an example of a speech recognition result. FIG. 2 shows an example of a speech recognition result for a speech I0 corresponding to Japanese meaning “Make an English course in the morning at MHK” (“Take a good practice even at Emuecchi-ke”). Is shown. FIG. 2 shows an example in which the speech recognition result is expressed in a lattice expression format.

  In this example, interpretation is performed between node 201 (“morning”) and node 202 (“morning”), and between node 203 (“English course”) and node 204 (“Go course”). There is an ambiguity.

  The numerical value attached to the line between the nodes of the lattice represents the cost calculated from the statistical co-occurrence frequency in the process of generating the lattice. In the figure, for example, the cost between node 205 ("MHK") and node 201 ("morning") is 2, node 202 ("morning") and node 203 ("English course") It is shown that the cost between

  The recognizing unit 111 generates a recognition candidate group including a predetermined number of candidates having higher likelihoods representing the likelihood based on the lattice expression and the cost of the recognition result. FIG. 3 is an explanatory diagram showing an example of the generated recognition candidate sentence. FIG. 3 shows the result of determining the ranking from the first candidate to the fourth candidate according to the likelihood corresponding to the total cost from the start node to the end node in FIG.

  As illustrated in FIG. 3, the recognition unit 111 generates a recognition candidate in which a candidate number for identifying a recognition candidate, candidate text representing the content of the recognition candidate, and likelihood are associated with each other. In the example of FIG. 3, the correct recognition result corresponding to the user's request is the third candidate. As described above, in the speech recognition processing, even if the first candidate is incorrect, a case where the correct recognition result is included in other candidates may occur.

  Returning to FIG. 1, the candidate generation unit 112 generates a corresponding response candidate for each recognition result candidate in addition to generating a response to the highest candidate in consideration of such a situation. . In addition, a response means the process performed according to a user's input audio | voice, or the content to output. Since this embodiment is an example of a video recording / playback apparatus, for example, processing such as playback / recording of a television program is a response. Hereinafter, a response is referred to as an action, and a response candidate is referred to as an action candidate.

  FIG. 4 is an explanatory diagram illustrating an example of an action. As shown in FIG. 4, the action includes four attributes (hereinafter referred to as action attributes) of “operation”, “date / time”, “channel”, and “program name”. Note that the second and subsequent rows in the table of FIG. 4 correspond to actions.

  For example, the second line represents the system operation of recording (operation) “English course” (program name) of “MHK” (channel) in “morning” (date and time). The third line represents an operation of reproducing “recording data 1”. Here, since “reproduction” represents an operation of immediate reproduction when a user request is made, the value of “date and time” is empty (represented by “−”). The value of “channel” is also empty.

  In this way, the action expression format is not fixed, and the process to be executed and the output contents may be represented by at least one word. In the example of FIG. 4, the content of the action can be specified if at least “operation” is set.

  The candidate generation unit 112 generates an action candidate group corresponding to the user's request by applying a language analysis method such as morphological analysis, syntax analysis, and semantic analysis to the recognition candidate group. At this time, the candidate generation unit 112 calculates the likelihood for each action candidate from the likelihood of each recognition candidate calculated in the speech recognition process and the certainty in the language analysis process, and ranks each candidate.

  FIG. 5 is an explanatory diagram illustrating an example of an action candidate group. FIG. 5 shows an example of action candidates for each recognition candidate shown in FIG. As shown in FIG. 5, the action candidate includes an identifier “candidate”, “operation”, “date / time”, “channel”, “program name”, and “likelihood” similar to those in FIG. 4. . In the table of FIG. 5, the second and subsequent lines correspond to actions, and are arranged in ascending order from Act1 which is the first candidate. In the example of FIG. 5, for the sake of simplicity, it is assumed that the language processing is performed correctly, and the likelihood value of the recognition candidate shown in FIG. 3 is used as it is as the likelihood value of the action candidate.

  Returning to FIG. 1, the response sentence generation unit 113 generates a response sentence for confirming to the user whether or not the action candidate having the maximum likelihood satisfies the user's request. Specifically, the response sentence generation unit 113 generates a response sentence using a template described by action attributes.

  FIG. 6 is an explanatory diagram illustrating an example of a template. As shown in FIG. 6, the template T includes a variable part designated by the symbol “{}” and other fixed parts. The variable part indicates that the attribute value of the corresponding action attribute of each action candidate is applied by designating the action attribute in the symbol “{}”. Further, the template T is divided by the phrase “/” so as to include one action attribute. The reason why the phrase is divided in advance in this way is to enable the output unit 102 described later to sequentially output response sentences in phrase units. In the following, a response sentence divided in units of phrases is referred to as a response phrase list, and is represented as P {P1 to PN} (N is the number of phrases).

  The method for generating a response sentence is not limited to the method using a template, and any conventionally used method such as a method for generating a sentence using a grammar rule or a generation rule can be applied.

  FIG. 7 is an explanatory diagram illustrating an example of a response phrase list generated using a template. FIG. 7 shows a response phrase list generated by applying the action candidate CAct1 of FIG. 5 to the template of FIG. The response phrases P1 to P4 are output from the output unit 102 in this order.

  Returning to FIG. 1, the corrected phrase generation unit 114 generates a corrected phrase indicating the correction contents of the response sentence spoken by the user with respect to the response sentence output by the output unit 102 described later. Specifically, the correction phrase generation unit 114 is an attribute corresponding to at least one of a plurality of action attributes constituting the action based on a recognition result candidate by the recognition unit 111 for the speech uttered for correction. Generate an action fragment containing the value as a modified phrase.

  When the user corrects the response sentence, the user may utter only the corrected portion instead of speaking the entire response sentence again. That is, the user's utterance may not include all action attributes (operation, date / time, channel, program name) of the action. Even in such a case, the corrected phrase generation unit 114 can extract at least a part of the action attributes from the recognition result candidates. Since the attribute value of the action attribute extracted in this way represents the correction content requested by the user, the corrected phrase generation unit 114 generates this attribute value as a corrected phrase.

  FIG. 8 is an explanatory diagram illustrating another example of the recognition candidate sentence generated by the recognition unit 111. FIG. 8 is a voice uttered by the user for requesting correction of the response sentence including the response phrase shown in FIG. 7, and uttered to correct “date and time” among the action attributes, “Morning”. Shows an example of a speech recognition result for Japanese input speech I1 ("Asadayo") meaning FIG. 8 shows that the only candidate (“Morning is”) is generated as a recognition result candidate.

  In response to such a recognition result, the corrected phrase generation unit 114 extracts information that the value of the action attribute “date” is “morning” as an action fragment. FIG. 9 is an explanatory diagram showing an example of the action fragment generated in this way. FIG. 9 is an example of an action fragment generated from the input voice I1 described above.

  Note that the modified phrase generation unit 114 and the candidate generation unit 112 differ only in whether an action fragment including only part of the action attribute or an action candidate including all of the action attributes is generated. That is, the processing procedure for interpreting the user's request by executing language analysis techniques such as morphological analysis, syntax analysis, and semantic analysis is common to the recognition result. Therefore, you may comprise so that either one may be integrated with the other.

  The selection unit 115 selects an action candidate group that includes all the action fragment attribute values from the action candidate group, and selects a candidate with the highest likelihood from the selected action candidate group as a new first candidate. It is.

  For example, it is assumed that an action candidate group as shown in FIG. 5 is generated and an action fragment (hereinafter referred to as action fragment SEG1) as shown in FIG. 9 is generated. In this case, the selection unit 115 searches the action candidate group in FIG. 5 for an action candidate whose attribute “date and time” matches the action fragment SEG1 ((morning) morning). In the example of FIG. 5, the selection unit 115 can acquire CAct3 and CAct4. Next, the selection unit 115 newly selects the one with the highest likelihood of CAct3 and CAct4 as the first candidate. In this example, since the likelihood of CAct3 = 0.2> the likelihood of CAct4 = 0.1, CAct3 is selected.

  The update unit 116 updates the response phrase list based on the action candidates selected by the selection unit 115. Specifically, the update unit 116 first determines between the action candidate newly selected by the selection unit 115 (hereinafter referred to as a new candidate) and the first action candidate before selection (hereinafter referred to as an old candidate). Compare all action attribute values. Then, the update unit 116 extracts a new candidate action attribute corresponding to the mismatched portion.

  FIG. 10 is an explanatory diagram illustrating an example of an old candidate. FIG. 11 is an explanatory diagram showing an example of a new candidate. In the example of FIGS. 10 and 11, the action attributes “date and time” and “program name” are different, and the update unit 116 extracts these action attributes.

  Next, the update unit 116 updates the response phrase corresponding to the extracted action attribute in the response phrase list generated from the old candidate with the new attribute value. In the example of FIG. 11, the update unit 116 acquires the attribute value 1101 ((the day) morning) and the attribute value 1102 (English course) as new attribute values. Then, the update unit 116 changes the content of the corresponding response phrase in the generated response phrase list with a new attribute value.

  FIG. 12 is an explanatory diagram showing an example of the response phrase list after being updated. FIG. 12 shows the response phrase list after the response phrase list of FIG. 7 is updated using the action candidate attributes as shown in FIG.

  As described above, the candidate generating unit 112 generates action candidates corresponding to all recognition result candidates in advance. For this reason, when the action is corrected, the selection unit 115 only needs to select a more appropriate action candidate from the generated action candidates according to the user's corrected utterance. That is, it is possible to simultaneously correct not only the response sentence (response phrase list) but also the action candidate according to the user's correction utterance for the response sentence.

  The output unit 102 generates a synthesized voice obtained by converting the response text generated by the response text generating unit 113 or the response text updated by the update unit 116 into a voice signal, and outputs the synthesized voice to the speaker 132. .

  Specifically, the output unit 102 first performs speech synthesis processing for converting each character string constituting the response sentence into a speech signal. For the speech synthesis processing by the output unit 102, any generally used method such as speech segment editing speech synthesis, formant speech synthesis, speech corpus-based speech synthesis, or the like can be applied. Then, the output unit 102 performs DA conversion on the generated audio signal and outputs it to the speaker 132.

  Further, when the response sentence is updated, the output unit 102 specifies from which part the updated response sentence is output. Specifically, the output unit 102 identifies a response phrase that has not been output in the response sentence before the update, and outputs a synthesized speech of the response sentence after the update from the identified response phrase.

  The recording / playback unit 103 executes the determined action, that is, the action candidate having the maximum likelihood. For example, when CAct3 in FIG. 5 is selected as the most likely action candidate, the recording / playback unit 103, according to each action attribute of CAct3, displays a program with a specified program name of a specified channel at a specified date and time. Perform the action to record.

  Note that a configuration unit that executes an actual action, such as the recording / playback unit 103, may be provided in the external device. In this case, information regarding the determined action is output from the voice interaction apparatus to the external apparatus, and the external apparatus is configured to execute the action with reference to this information.

  Next, a voice interaction process performed by the video recording / playback apparatus 100 according to the present embodiment configured as described above will be described with reference to FIG. FIG. 13 is a flowchart showing the overall flow of the voice interaction process in the present embodiment.

  First, the reception unit 101 determines whether or not the input voice I0 is input from the microphone 131 (step S1301). If the input voice I0 has not been input (step S1301: NO), the process is repeated until it is input.

  When the input voice I0 is input (step S1301: YES), the recognition unit 111 recognizes the input voice I0 and generates a recognition candidate group (step S1302). Next, the candidate generation unit 112 obtains a corresponding action candidate for each candidate of the recognition candidate group, and generates an action candidate group CAct {CAct1 to CActM} (M is the number of action candidates) (step S1303).

  Next, the response sentence generation unit 113 determines the action candidate ACT having the maximum likelihood (step S1304). Next, the response sentence generation unit 113 generates a response phrase list P {P1 to PN} (N is the number of phrases) corresponding to the action candidate ACT (step S1305). Specifically, the response sentence generation unit 113 refers to the template as shown in FIG. 6 and applies the attribute value of the corresponding action attribute of the action candidate ACT to the variable part of the template, respectively, so that the response phrase list P Is generated.

  Next, the output unit 102 sequentially obtains response phrases Pi (i = 1 to N) from the generated response phrase list P, and outputs synthesized speech obtained by speech synthesis (step S1306). Note that i is a counter value indicating the output order of response phrases.

  Next, the reception unit 101 determines whether or not the input voice Ii is input from the microphone 131 (step S1307). The input voice Ii means that the voice is input during the output of the i-th response phrase Pi, but it is not necessarily a voice that indicates the correction content of the response phrase Pi. That is, the correction contents of any of the response phrases P1 to Pi-1 output before the response phrase Pi may be represented. In addition, if the user guesses unspoken response phrases Pi + 1 to PN and speaks, the input voice Ii may represent the correction contents of the response phrases Pi + 1 to PN.

  When the input voice Ii is input (step S1307: YES), candidate update processing for updating the most likely action candidate and the corresponding response sentence is executed according to the contents of the input voice Ii (step S1308). Details of the candidate update process will be described later.

  After the candidate update process or when the input voice Ii is not input in step S1307 (step S1307: NO), the output unit 102 determines whether all response phrases have been processed (step S1309).

  If all response phrases have not been processed (step S1309: NO), the output unit 102 repeats output processing for the next response phrase (step S1306). As will be described later, when the action candidate is changed in the candidate update process, the response sentence (response phrase list) is updated corresponding to the action candidate after the change. Obtain the next response phrase from the response phrase list and output it.

  When all the response phrases have been processed (step S1309: YES), the recording / playback unit 103 executes an action corresponding to the most likely action candidate ACT (step S1310).

  In this way, a response sentence for confirming the content of the action that is a response to the user's request is generated, and when a sound for correction is input during the output of the response sentence, the action and Response sentences can be changed at the same time. Thereby, the dialog with the user can be smoothly advanced while the error part can be easily corrected by voice.

  Next, details of the candidate update process in step S1308 will be described with reference to FIG. FIG. 14 is a flowchart showing the overall flow of candidate update processing in the present embodiment.

  First, the recognition unit 111 recognizes the input voice Ii and outputs a recognition result (step S1401). Next, the corrected phrase generation unit 114 analyzes the recognition result and generates an action fragment group SEG {SEG1 to SEGK} (K is the number of action fragments) including an attribute value of at least one action attribute (step S1402). .

  Next, the selection unit 115 determines whether or not the action fragment group SEG exists (step S1403). If it exists (step S1403: YES), it corresponds to the same action attribute as the element of the action fragment group SEG. Select action candidates whose attribute values match for all elements. Then, the action candidate CActk having the maximum likelihood is selected from the selected action candidates (step S1404).

  Next, the selection unit 115 determines whether or not there is an action candidate CActk (step S1405). When the action candidate CActk exists (step S1405: YES), the update unit 116 compares the action candidate CActk (new candidate) with the current maximum likelihood action candidate ACT (old candidate). Then, the update unit 116 generates a mismatch attribute group Att {Att1 to AttL} (L is the number of mismatch attributes) including a new candidate action attribute (hereinafter referred to as a mismatch attribute) corresponding to the mismatch part (step S1406). .

  Next, the selection unit 115 determines whether or not the mismatch attribute group Att exists (step S1407), and if it exists (step S1407: YES), sets the action candidate CActk as the most likely action candidate ACT. (Step S1408).

  Next, the update unit 116 replaces the response phrase corresponding to the action attribute included in the mismatch attribute group Att in the response phrase list P with the attribute value of the mismatch attribute group Att (step S1409).

  Subsequently, the output unit 102 executes the following processing to specify from which response phrase the updated response phrase list P is to be output (steps S1410 to S1412).

  First, the output unit 102 acquires the position j from the beginning of the attribute value closest to the beginning of the replaced attribute values (step S1410). Next, the output unit 102 determines whether the position j of the acquired attribute value is before the position i of the response phrase already output in the response phrase list P before update (step S1411).

  Normally, correction contents for the output response phrase are spoken and the corresponding attribute value is replaced, so j is smaller than i. However, j may not be smaller than i, for example, when the user guesses a response phrase as described above and correction contents for an unoutput response phrase are spoken.

  When the position j is before the position i (step S1411: YES), the output unit 102 sets the position j of the replaced attribute value as the next output position (step S1412). That is, the output unit 102 substitutes j for i.

  When it is determined in step S1403 that the action fragment group SEG does not exist (step S1403: NO, in step S1405, it is determined that the action candidate CActk does not exist (step S1405: NO), the mismatch attribute group Att is determined in step S1407. If it is determined that it does not exist (step S1407: NO), or if it is determined in step S1411 that the position j is not before the position i (step S1411: NO), the candidate update process is terminated.

  Next, a specific example of voice dialogue processing by the video recording / playback apparatus 100 according to the present embodiment will be described.

  First, in the morning of the day, in the morning of the day, “MHK” in the channel called “English lecture” is set for the purpose of setting a recording reservation for a program in Japan. The word input voice I0 (Emu-Ecchi-Ke, take a look at the word) is input (step S1301). Subsequently, the recognition unit 111 recognizes the input voice I0 and generates a recognition candidate group as shown in FIG. 3 (step S1302). Further, the candidate generation unit 112 generates an action candidate group CAct shown in FIG. 5 from this recognition candidate group (step S1303).

  Note that, as described above, in the example of FIG. 3, the action candidate that meets the user's request is the third candidate.

  In the action candidate group CAct, the candidate having the highest likelihood is CAct1 having a likelihood of 0.4, and therefore CAct1 is set to ACT (step S1304). Next, the response sentence generation unit 113 corresponds to a variable part of a template T as shown in FIG. 6 (/ {date} broadcast on {channel} / {program name}). The attribute value of the action attribute corresponding to CAct1 is inserted into each of the action attributes to generate a response phrase list P (step S1305). FIG. 7 shows a response phrase list P generated at this time.

  Next, the output unit 102 synthesizes and outputs a response phrase P1 (in MHK) corresponding to the counter i (= 1) (step S1306). Here, it is assumed that the input voice I1 is not input from the user during the output process of the response phrase P1 (step S1307: NO). Subsequently, the output unit 102 synthesizes and outputs a response phrase P2 (broadcasted tomorrow) corresponding to the next counter i (= 2) (step S1306).

  Here, it is assumed that during the voice output of the response phrase P2, the user notices that the designation of the date and time (the morning of (today)) of the first input voice I0 is misinterpreted. Then, it is assumed that the user has input a Japanese input voice I2 (Asadayo) meaning “It is morning” in order to correct the recording date and time in the morning (step S1307: YES).

  In this case, candidate update processing for updating the most likely action candidate ACT and the response phrase list P based on the input speech I2 is executed (step S1308).

  In the candidate update process, first, the recognition unit 111 recognizes the input voice I2 and generates a recognition candidate group as shown in FIG. 8 (step S1401). Further, the corrected phrase generation unit 114 generates an action fragment group SEG corresponding to the recognition candidate group (step S1402). Here, since only the information of the attribute “date and time” of the action candidate is extracted, the action fragment group SEG {SEG1} is obtained.

  Subsequently, the selection unit 115 selects from the action candidate group CAct an action candidate group whose attribute “date and time” is “(morning) morning” of the element of the action fragment group SEG (here, only SEG1). In this example, the selection unit 115 selects CAct3 and CAct4 in FIG. And the selection part 115 makes CAct3 (likelihood 0.3) with the largest likelihood among these candidates the maximum likelihood candidate CActk (step S1404).

  Since the maximum likelihood candidate CActk is found (step S1405: YES), the updating unit 116 compares the attribute values of CAct3 and ACT (= CAct1) to generate a mismatch attribute group Att (step S1406). In this example, as shown in FIG. 11, the action attribute “date” corresponding to the attribute value 1101 and the action attribute “program name” corresponding to the attribute value 1102 are included in the mismatch attribute group Att.

  Therefore, the updating unit 116 responds to the corresponding attribute values ({the day after tomorrow}) and {the day after tomorrow} with the response phrase list P ({MHK} / {the day after tomorrow} broadcast / {go course}}). (Course}) is replaced with the attribute value of CAct3 (“morning” and “English course”) (step S1409). FIG. 12 shows the response phrase list P updated in this way.

  Through the processing so far, the input speech uttered by the user in response to the response sentence is fed back, and the action and the response phrase corresponding to the action can also be corrected.

  However, when the response phrase is corrected, the user's convenience varies greatly depending on whether the response sentence (response phrase list) output halfway is output again from the beginning or only the corrected portion is output.

  Therefore, in the present embodiment, as described above, a part that has already been output in the response sentence is configured to be output as much as possible while avoiding re-output as much as possible. That is, if the response phrase Pj closest to the beginning of the sentence among the updated response phrases (the response phrase with the smallest subscript j) has already been output, the output unit 102 resumes output from the response phrase Pj. If the response phrase Pj is not output, the output unit 102 continuously outputs the response phrase Pi indicated by the counter i indicating the current output position.

  In the above example, the updated response phrase closest to the beginning of the sentence is P2 (broadcast {morning}). That is, the smallest subscript j among the subscripts of the updated response phrase is 2, which matches the current counter i = 2, so the counter i is not updated (step S1411: NO).

  Thereafter, the output unit 102 outputs the synthesized speech of the updated response phrase P2 (broadcasted in {morning}) (step S1306). Here, it is assumed that the user confirms that the input speech I2 has been correctly interpreted by listening to the synthesized speech, and has not made an utterance for correction.

  Thereafter, similarly, the response phrase P3 ({English course}) and the response phrase P4 ({record}?) Are sequentially output. If a response utterance from the user is not detected during that time, the confirmed action is executed by the recording / playback unit 103 after the response text is output (step S1310). Thereafter, the process returns to the state of accepting input from the user (step S1301).

  Thus, in the voice interaction apparatus according to the present embodiment, response phrases corresponding to the user's requested utterance are sequentially output, and when there is a response for correction from the user, action candidates and response phrase lists are displayed. It can be corrected at the same time. Moreover, since the utterance of the response phrase is continued from the corrected part, the output of the part that has been output before the update can be omitted. As a result, it is possible to realize a voice dialogue device that can be easily corrected without obstructing the dialogue by taking extra steps.

  Even if the user who has heard the voice of the response sentence guesses the error about the part that has not been output yet and rephrases it, the correction part can be identified and an appropriate candidate can be selected again. . As a result, the convenience of the user can be improved and the conversation can proceed more smoothly.

(Modification)
In the above embodiment, response phrases are generated according to a fixed template as shown in FIG. 6, and the generated response phrases are sequentially output.

  However, if the response phrase near the beginning of the sentence is incorrect, there is little information that is output up to the point in time when the incorrect response phrase is output. There may be cases where it cannot be judged properly.

  For example, it is assumed that “MHK”, which is the channel name of the first response phrase P1 (with {MHK}) in the response phrase list of FIG. 7, is an error of “LHK”. However, when the response phrase P1 is output, it is not always possible for the user to instantaneously determine that the response phrase is a response phrase corresponding to the channel name from only the fragmentary information.

  Therefore, in this modification, such a problem is reduced by outputting a response phrase with less ambiguity of interpretation first. However, if response phrases are simply rearranged in the order of less ambiguity, a response sentence with an unnatural meaning or a grammatically inappropriate response sentence may be generated due to linguistic restrictions.

  For example, when the response sentence corresponding to FIG. 7 is rearranged as “Broadcast tomorrow / MHK / Go course / Record?”, “Broadcast” is related to “MHK”, meaning Will result in an incorrect response.

  Therefore, a restriction rule for rearrangement is constructed, and a response phrase list is generated according to the rule. For example, a plurality of templates covering patterns that can be rearranged are prepared in advance, and an optimum template is selected to generate a response sentence. Specifically, the response sentence generation unit 113 selects an optimal template from such a template according to ambiguity and applies the attribute value of the most likely action candidate to generate a response sentence.

  FIG. 15 is an explanatory diagram showing an example of a template used in this modification. FIG. 15 shows an example of four templates with different response phrase output orders.

  For example, when the action candidate group of FIG. 5 is generated and a response sentence of the maximum likelihood action candidate CAct1 is generated, first, the response sentence generation unit 113 determines the ambiguity of each action attribute of the action candidate. In the example of FIG. 5, since the action attributes “operation” and “channel” have only one attribute value, it is determined that the ambiguity is low. The action attributes “date and time” and “program name” have two attribute values, respectively, and thus are determined to have high ambiguity.

  Therefore, the response sentence generation unit 113 preferentially selects a template in which the action attributes “operation” and “channel” appear first. In the example of FIG. 15, the response sentence generation unit 113 selects the template T2 ({Do you want to operate}? / {Channel} / {Date} broadcast / {Program name} /). In this case, the response sentence generation unit 113 generates “{record}} / {MHK} / {broadcast tomorrow} / {go program} /” as the response phrase list.

  In this way, since the response sentence is generated according to a predetermined template, a grammatically incorrect response sentence is not generated. Further, since the response phrases are output in order from the less ambiguous response phrases, there is a high possibility that a lot of information (response phrases) will be output before the erroneously recognized response phrases are output. As a result, it is possible to solve the above-described problem that the suitability of the response phrase cannot be properly determined due to the small amount of information.

  Next, the hardware configuration of the voice interaction apparatus according to the present embodiment will be described with reference to FIG. FIG. 16 is an explanatory diagram showing a hardware configuration of the voice interaction apparatus according to the present embodiment.

  The voice interactive apparatus according to the present embodiment includes a communication I / O that communicates with a control device such as a CPU (Central Processing Unit) 51 and a storage device such as a ROM (Read Only Memory) 52 and a RAM 53 by connecting to a network. F54 and a bus 61 for connecting each part are provided.

  The voice interaction program executed by the voice interaction apparatus according to the present embodiment is provided by being incorporated in advance in the ROM 52 or the like.

  The voice dialogue program executed by the voice dialogue apparatus according to the present embodiment is a file in an installable format or an executable format, and is a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R. (Compact Disk Recordable), DVD (Digital Versatile Disk) or the like may be provided by being recorded on a computer-readable recording medium.

  Furthermore, the voice dialogue program executed by the voice dialogue apparatus according to the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. . Further, the voice dialogue program executed by the voice dialogue apparatus according to the present embodiment may be provided or distributed via a network such as the Internet.

  The voice dialogue program executed by the voice dialogue apparatus according to the present embodiment has a module configuration including the above-described units (accepting unit, dialogue processing unit, output unit, recording / playback unit), and as actual hardware. When the CPU 51 reads out and executes the voice interaction program from the ROM 52, the above-described units are loaded onto the main storage device, and the respective units are generated on the main storage device.

  As described above, the apparatus and method according to the present invention are suitable for a video recording / playback apparatus, a car navigation system, a game machine, and the like that operate in response to a request input by voice.

It is a block diagram which shows the structure of the video recording / reproducing apparatus concerning this Embodiment. It is explanatory drawing which shows an example of a speech recognition result. It is explanatory drawing which shows an example of a recognition candidate sentence. It is explanatory drawing which shows an example of an action. It is explanatory drawing which shows an example of an action candidate group. It is explanatory drawing which shows an example of a template. It is explanatory drawing which shows an example of a response phrase list. It is explanatory drawing which shows another example of a recognition candidate sentence. It is explanatory drawing which shows an example of an action fragment. It is explanatory drawing which shows an example of an old candidate. It is explanatory drawing which shows an example of a new candidate. It is explanatory drawing which shows an example of the response phrase list | wrist after being updated. It is a flowchart which shows the whole flow of the voice dialogue process in this Embodiment. It is a flowchart which shows the whole flow of the candidate update process in this Embodiment. It is explanatory drawing which shows an example of the template utilized in a modification. It is explanatory drawing which shows the hardware constitutions of the voice interactive apparatus concerning this Embodiment.

Explanation of symbols

51 CPU
52 ROM
53 RAM
54 Communication I / F
61 Bus 100 Video Recording / Playback Device 101 Reception Unit 102 Output Unit 103 Recording / Playback Unit 110 Dialogue Processing Unit 111 Recognition Unit 112 Candidate Generation Unit 113 Response Sentence Generation Unit 114 Corrected Phrase Generation Unit 115 Selection Unit 116 Update Unit 120 Storage Unit 131 Microphone 132 Speaker 201-205 Node 1101, 1102 Attribute value

Claims (6)

A recognition unit that recognizes input speech and generates a plurality of recognition result candidates;
A plurality of first recognition result candidates for the first speech are analyzed, and a response candidate corresponding to each of the plurality of first recognition result candidates and a likelihood representing a probability of a response candidate for the first recognition result candidate are expressed. A candidate generator for generating degrees;
The first candidate of the first recognition result including a phrase representing the candidate of the response to the first candidate of the selected first recognition result is selected from the candidate of the response to the first candidate of the first recognition result having the maximum likelihood. A response sentence generator for generating a response sentence for the candidate;
An output unit that outputs a synthesized speech obtained by converting a response sentence to the first candidate of the first recognition result into a speech signal;
When a second voice is input during the output of the synthesized voice, a second recognition result candidate for the second voice generated by the candidate generation unit is analyzed, and a response to the first candidate of the first recognition result A corrected phrase generation unit that generates a corrected phrase by correcting a phrase included in the sentence;
A response candidate for another candidate of the first recognition result including the same phrase as the corrected word is obtained from response candidates for the plurality of first recognition result candidates, and a response of another candidate of the first recognition result is obtained. A selection unit for selecting a candidate for a response to another candidate of the first recognition result having the maximum likelihood among the candidates;
An update unit that updates the response sentence with a candidate word of a response to another candidate of the selected first recognition result,
When the response sentence is updated, the output unit outputs the synthesized voice of the response sentence after the update instead of the synthesized voice of the response sentence before the update,
A voice interaction device characterized by the above. The output unit, when the response sentence is updated, outputs the synthesized speech of the updated response sentence from a phrase corresponding to a phrase that has not been output in the response sentence before update,
The voice interactive apparatus according to claim 1. The output unit is updated when a phrase that has been output in the response sentence before the update is included at the end of the sentence from the phrase that is closest to the beginning of the updated phrase among the phrases included in the response sentence. Outputting the synthesized speech of the response sentence updated from the phrase closest to the beginning of the phrase,
The voice interactive apparatus according to claim 2. The output unit is output when the phrase that has been output in the response sentence before the update is included in the response sentence before update from the phrase that is closest to the beginning of the updated phrase. Outputting the synthesized speech of the response sentence updated from the phrase included at the end of the sentence next to the phrase of
The voice interactive apparatus according to claim 2. The candidate generation unit further generates a second likelihood that represents a probability for each word that represents the response candidate,
The response sentence generation unit generates the response sentence including words representing the response candidates from the beginning of the sentence in order of increasing second likelihood;
The voice interactive apparatus according to claim 1. A recognition step of recognizing input speech by the recognition unit and generating a plurality of recognition result candidates;
The candidate generation unit analyzes a plurality of first recognition result candidates for the first speech, and selects response candidates corresponding to the plurality of first recognition result candidates, and response candidates for the first recognition result candidates. A candidate generating step for generating a likelihood representing the likelihood;
A response sentence generator selects a candidate for a response to the first candidate of the first recognition result with the maximum likelihood, and includes a phrase that represents a candidate for a response to the selected first candidate of the first recognition result. A response sentence generation step for generating a response sentence for the first candidate of one recognition result;
A first output step of outputting a synthesized speech obtained by converting a response sentence to the first candidate of the first recognition result into a speech signal by the output unit;
When a second speech is input during the output of the synthesized speech by the correction word generation unit, the second recognition result candidate for the second speech generated in the candidate generation step is analyzed, and the first recognition result A corrected phrase generation step of generating a corrected phrase by correcting the phrase included in the response sentence to the first candidate of
The selection unit acquires a response candidate for another candidate of the first recognition result including the same phrase as the corrected phrase from the response candidates for the plurality of first recognition result candidates, A selection step of selecting a candidate for a response to another candidate of the first recognition result having the maximum likelihood among candidates for a response to the candidate;
An update step of updating the response sentence with a word of a candidate for a response to another candidate of the selected first recognition result by the update unit,
A second output step of outputting the synthesized speech of the response sentence after the update instead of the synthesized speech of the response sentence before the update when the response sentence is updated by the output unit;
A voice dialogue method characterized by comprising:

Images