JP2000259177A - Voice outputting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a voice outputting device which outputs most desirable smooth voices for a user by dynamically controlling output voices in accordance with the situation in which the user is placed. SOLUTION: An input.recognition section 101 recognizes voices uttered by a user. A dialogue processing section 102 selects a voice that corresponds to the dialogue status at that time from the output data registered in an output table 103. A deformation section 104 deforms the words to be assembled into the output data on the basis of the voices from the user and a voice output section 109 vocally outputs the words. The section 104 refers to a deformation contents table 108, an adding word table 105 for characters, an adding word table 106 for words and a deformation counter 107.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio output device used when interacting with a speaker or when recognizing and outputting the contents spoken by the speaker.

[0002]

2. Description of the Related Art A voice output device used for a dialogue with a speaker recognizes an input voice and outputs a recognized voice for confirmation thereof, or a device for question or answer. And output the generated audio. In this case, in order to improve the efficiency of the dialogue and secure communication between the speaker and the device, emphasis processing such as increasing the volume is performed on important parts, and difficult words are input. In such a case, a method is employed in which the data is replaced with a plain expression and voice is output for confirmation. For example, Japanese Patent Application Laid-Open No. 10-1714
The device disclosed in Japanese Patent Publication No. 85 is designed to analyze text information to identify an important part, apply a transformation such as emphasis to the important part, and output the modified part. Also, Japanese Patent Application Laid-Open No. 7-121
The device disclosed in Japanese Patent No. 537 discloses a device that specifies an esoteric word or phrase from a text to be read and replaces it with a plain expression and reads the word.

[0003]

However, in the above-described apparatus, since the processing according to the past output state is not performed,
When the same output is provided, the transformation process is always performed, and as a result, a sound output that the user feels troublesome may be obtained.

Since it is not considered that an error occurs in the result of speech recognition on the device side with respect to the utterance of the user, it is difficult for the user to detect a recognition error of the device.

[0005] In order to replace homonyms and the like with plain expressions, expressions for explanation must be prepared for all words used.

There have been problems such as the following.

An object of the present invention is to provide an audio output device capable of performing the most desirable smooth audio output for a user by dynamically controlling the output audio according to the situation of the user.

[0008]

SUMMARY OF THE INVENTION According to the present invention, the user's situation, such as the user's confusing, interrupt occurrence, and the certainty of recognition of the input speech from the user, is known, and the output speech is dynamically generated according to the occasional situation. To control.

The following terms are used in the present invention.

Speech recognition: Recognizing each phoneme by decomposing a user's input speech into phonemes. -Dialogue status: In a series of dialogues between the user and the device,
Each stage of the dialogue, such as "greeting", "confirmation", "answer", etc.

Expressions that express confusingness: Expressions of voices and attitudes expressed by the user when the user cannot recognize the output of the device.

[0012] Deformation of words incorporated in output data:
This refers to changing the manner of outputting words incorporated in output data that is output as voice from the device to the user.

[0013] Interruption voice of the user: A voice uttered by the user to give some interruption during output from the apparatus side.

Recognition certainty of input voice: The certainty of recognition when recognizing a user's input voice by voice.

Additional words: words added immediately before or immediately after to describe words, characters, and the like incorporated in output data from the device. The present invention is configured as follows.

(1) A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each conversation state with a user, and a voice recognition unit An output data selecting means for determining a dialogue state from the result and selecting output data to be output at that time from the output data table, an audio output unit for outputting audio of the selected output data, Means for detecting a voice representing a confusing user from a voice recognition result, and deforming means for deforming a word incorporated in output data when detecting the voice indicating the confusing and re-outputting the word at the voice output unit. (Claim 1).

The voice input unit is composed of, for example, a microphone. A voice recognition unit that performs voice recognition from input voice detects a phoneme of voice input by the user. For example,
When the user speaks “I”, the voice recognition unit recognizes “Wa”, “Ta”, and “Shi” as each phoneme. This speech recognition method includes extracting a feature pattern such as a linear prediction coefficient or a Fourier spectrum coefficient from speech data, and matching the pattern with a pattern stored in a dictionary to perform recognition. Hidden (H
A well-known method such as a speech recognition method using a Markov method or a syntax recognition method is used. In the pattern matching method, a DP (dynamic programming) pattern matching method for absorbing a difference in utterance speed depending on a user is generally adopted.
In speech recognition, words and parts of speech can be recognized by performing morphological analysis together with recognition of each phoneme. In the present invention, the recognition of words and parts of speech may or may not be performed by the voice recognition unit.

The output data table stores data to be output as voice from the apparatus for each situation of conversation with the user. The dialogue state means, for example, a state in a dialogue in which a greeting is first given in a series of dialogues, then a voice uttered by the user is confirmed, and a question from the user is recognized and answered. By storing in advance in the output data table for each dialogue situation, it is possible to deal with any scene by updating the contents of this table. Note that the output data is combined with fixed data and embedded data (words and the like). The fixed data is voice pattern data fixed in advance on the system, and the embedded data is distinguished into voice pattern data for returning voice input from a user as it is and voice pattern data dynamically generated in the system. . Normally, when the conversation status is "confirmation" type, the embedded data is a voice pattern that returns the input voice from the user as it is, and when the conversation status is "answer" type, the embedded data is dynamically generated. It becomes the voice pattern.

The output data selection means determines a dialogue state from the speech recognition result, and selects output data to be output at that time from the table. The output means outputs the selected output data from a speaker or the like. At this time,
Embedded data in the output data is generated according to the conversation situation.

When the user detects a voice that indicates a confusion, such as "eh," with respect to the output voice output by the output means, the embedded data included in the output data is transformed and output again. Toxic is an expression that utters as a result of the inability of the user to recognize the output of the device. It is a form that expresses confusing expressions such as "Nani" in addition to "Eh". When this confusing is detected, for example, it is output again in response to the deformation of “volume higher” and “speed lower” than the previous time. If no confusing speech is recognized, the next dialogue is entered.

(2) A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each conversation state with a user, An output data selecting means for determining a dialogue state from the result and selecting output data to be output at that time from the output data table, an audio output unit for outputting audio of the selected output data, Means for detecting a user's interrupt voice from a voice recognition result; and deforming means for deforming a word incorporated in output data when the interrupt voice is detected and outputting the word again at the voice output unit. (Claim 2).

According to the present invention, if the interrupt request from the user is voice, the data (words or the like) incorporated in the output data is transformed and output again. The interrupt sound includes, for example, sounds such as "retry" and "more detailed".

(3) A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each situation of dialogue with a user, Output data selecting means for judging a dialogue state from the result and selecting output data to be output at that time from the output data table; an audio output unit for outputting the selected output data as audio; and an imaging means for imaging the user Means for detecting a user's confusing state with respect to the audio output from the apparatus from the user's image captured by the imaging means; and deforming a word incorporated in output data when the confusing state is detected from the user's image. And a transforming means for causing the sound output unit to re-output the sound (claim 3).

According to the present invention, the user's confusing state is detected from the user image picked up by the image pickup means. For example,
When the user spreads both hands and takes a pose in which the meaning of the voice output emitted by the device cannot be understood, it is determined that the user is stuck. In the image processing in this case, an image corresponding to both hands of the user is focused on, and when it is detected that the both hands are spread left and right, it is determined that the state becomes confusing.

(4) A voice input unit for inputting a voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each conversation state with a user, An output data selecting means for judging a dialogue state from the result and selecting output data to be output at that time from the output data table; an audio output unit for outputting the selected output data as an audio; And a transformation means for transforming a word incorporated in the output data when the certainty is equal to or less than the threshold value and causing the word to be outputted again by the voice output unit (claim 4).

According to the present invention, when the input voice of the user cannot be reliably recognized due to the surrounding noise situation or the like, the word incorporated in the output data is modified. The present invention is applied to the case of confirming the content of a voice uttered by a user.
For example, if the user inputs a voice "Osaka no Osamu Osamu", the device outputs output data of "Okay?", But at this time, the recognition certainty of the input voice from the user is threshold. If the value is equal to or less than the value, the word “Osaka” to be incorporated in the output data is output slowly and surely. Alternatively, the volume is increased and output. As a result, the user can correctly know the degree of recognition by the device-side recognition means, and if it is not correct, gives a positive response such as "Yes", and if it is wrong, returns a negative response such as "No". Will do. When the device recognizes the voice input of the positive response, it enters the next dialogue, and when it recognizes the voice input of the negative response,
A modified output operation such as repeating the audio output at a lower speed or repeating the audio output with the volume further increased is performed.

(5) A word-by-word additional word table for storing additional words for explaining words to be incorporated in the output data, wherein the deforming means outputs a speech in which the additional words are added to the word to transform the words. (Claim 5).

In the modification of the voice incorporated in the output data, it is possible to output the voice by adding an additional word in order to make the contents of the output voice more understandable. The additional word is a term for explaining a word incorporated in the output data. For example, if the target word is "Hashi", "Bridge" is added to the word as an additional word. Therefore, in this case, "bridge bridge" is output. A case where the additional word is added is performed when the user inputs a voice such as “eh” which indicates a confusing situation.

(6) An additional word table for storing additional words for each character for explaining each character of the word incorporated in the output data, wherein the deforming means adds an additional word to each character constituting the word. The sound is transformed by outputting the sound (claim 6).

As the additional words, there are additional words for each character constituting the word, in addition to the additional words for the word as described in (5) above. For example, when performing the audio output of “Tokyo”, “Higashi Toto” is added as an additional word. Therefore, as a whole, the voice is output as "Higashi no Toyo Tokyo". Such a modification is performed, for example, when the user inputs a voice such as "eh" even if the additional word is added to the word and the voice is output in (5). In other words, when it is considered that it is easier for the user to understand that it is easier for the user to comprehend the word and to output the voice by adding the additional word for each character constituting the word than to output the voice by adding the additional word to the word. Deformation is performed.

(7) A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, and means for generating a voice recognition result as output data in order to repeat the input voice on the device side. A voice output unit that outputs output data as voice, a word-specific additional word table that stores additional words for explaining words to be incorporated in the output data, and the incorporated words to which the additional words have been added. And a deforming means for deforming and re-outputting by the output means (claim 7).

The present invention is applied to a voice input system for recognizing all sentences of a user's voice. For example, this apparatus is used when a user reads a document written on paper and converts it into digitized data. That is, the user reads out a certain amount of the original, and the apparatus recognizes the voice and outputs the same content as voice. At this time, if there is a homonym, an additional word is added to the word that is output by the device. Notation of a word that can be understood by adding an additional word, that is, if the kanji is different, the user inputs a correction by voice. By repeating this, digitized data can be created only by voice input.

(8) After the output data in which the transformed word is incorporated is output, when a word to which an additional word is added is spoken by the user as a corrected word, the corresponding word is corrected by the input correction. Means for converting to words, and when the additional words spoken by the user in the word-specific additional word table are unregistered, the homonymous words and the additional words are sequentially
A table updating unit that outputs a voice as a candidate and, when there is an input indicating that the word is a correct additional word from the user, adds the corresponding word to the word-based additional word table as a candidate additional word at that time. (Claim 8).

According to the present invention, in the apparatus of the above (7), when a word to which an additional word is added is input as a corrected word by the user, the corresponding word is converted into the input corrected word and output again. . At this time, if the additional word input by voice from the user is unregistered, the word of the homonymous word and the additional word are sequentially output as candidates. When the correct additional word is output as a voice, the user inputs a voice (for example, “Yes”) indicating that it is a correct additional word, and at this time, the candidate additional word is newly stored in the table. Through this series of operations, the word-by-word additional word table is automatically learned, so that a voice output device having a higher capability as it is used can be provided.

(9) When the word incorporated in the output data is transformed by the transformation means and output by the audio output unit, a ramp, buzzer,
A notifying means for notifying by means other than voice such as a vibrator is provided (claim 9).

According to the present invention, when a word incorporated in the output data is deformed and re-outputted, when the incorporated word is output, a notification is provided by means other than voice such as a lamp, a buzzer, or a vibrator. In particular, when important information or voice recognition of the user is uncertain, the user can be easily informed of points to be emphasized especially to the user.

[0037]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows an image diagram of a scene using an audio output device according to an embodiment of the present invention. This audio output device indicates an information terminal installed in a street, a government office, a hotel, or the like. Instead of such an information terminal, the voice output device can be applied to a voice interactive system using a telephone or the like.

In the figure, when the user approaches the voice output device, a voice is output from the voice output device saying "Welcome. It is a tourist information center." This audio output is
Because it is insignificant information, it is output quickly and quickly. When the user hears this guidance, for example, he hears "What is a recommended place in Nagaokakyo City?" After recognizing the voice output from the user, the voice output device determines the dialogue state, selects a voice output pattern (output data) to be output at that time from a table, and outputs the voice. At this time, if the voice of the user of "Nagaokakyo-shi" is not sufficiently recognized, "Nagaokakyo" is memorized slowly and surely, and the subsequent "Is it OK?" That is, when the recognition certainty of the voice recognition is below a certain threshold value, it is determined that the user is not confident in the voice recognition and important information (as will be described later, this information is a word incorporated in the entire output data). Is transformed and output so as to be surely confirmed by the user.

The above "Nagaoka Kyoshi" from the device
(Slow and reliable output) In response to the "Is it OK" (quickly and quickly output) output, the user utters "Yes" as a voice that indicates affirmation, and when this is recognized, the device turns to "Komiji" (Slowly and reliably output important information), but "Recommended" (quickly and quickly output as unimportant information). At this time, the user
If you do not know "Komiji", "Eh" is uttered as a voice that is confusing. The device is this "eh"
Is recognized, it is regarded as not knowing, and this is treated as an obfuscated word, an additional word is added and output again. The additional words include an additional word that describes the word itself and an additional word that describes each character constituting the word.
In the example shown in FIG. 1, the characters that make up the word "Kyoumoji" are "Kou", "Myou", and "Ji", and these characters are described using additional words. In other words, the device outputs “Hikaru Akari no Tera no Komoji” (slow and reliable output) and “recommended” (fast and quick output).

In the above example, "Kyoumyoji" is treated as an obfuscated word, and is re-output by adding an additional word for each character. Also, when a homonym is output instead of an obfuscated word, a word or an additional word of each character constituting the word is added and output again. In this way, the user interacts with the device side, and the device side selects and outputs speech data stored in advance in the table for each situation, and also outputs words incorporated in the speech data at the time. And output it. Note that words and the like incorporated into the entire output data include those that directly return the user's input and those that are dynamically generated by the device itself. In the example shown in FIG.
“Nagaoka” is a word that returns an input by the user, and “Komiji” is a word dynamically generated on the device side.

As will be described later, whether a word to be incorporated into the entire output data is a word for returning an input by a user or a word dynamically generated by an apparatus is stored in advance for each conversation situation. For example, if the dialogue situation is of the initial "greeting" type, the output data is fixed and no words are embedded therein. In the example above,
"Welcome. This is a tourist information center." Is the output data when this type is "greeting". When the conversation situation is of the “confirmation” type, the word incorporated in the entire output data is a word that returns an input by the user. In the example illustrated in FIG. 1, the word includes “Nagaoka Kyoshi” of “Is it OK?”. Further, when the dialogue situation is of the “simple answer” type, the words incorporated in the entire output data are words dynamically generated on the device side. In the example shown in FIG.
Of the output data of "Kyouji is recommended.", This is a word that incorporates "Kyouji" and is a part that is dynamically generated on the device side.

FIG. 2 shows a system configuration diagram of the audio output device. An input device 1 composed of a microphone is used as a voice input unit, and an output device 2 composed of a speaker is used as output means for outputting output data as a voice output pattern. The input device 1 and the output device 2 exchange data with an operating system 3 that controls a dialog system. The operating system 3 includes a dialog processing application program 4 that performs a dialog process based on a voice output program. Is implemented. External recording medium 5
Is the interactive application 4
Or provide data. In addition, by using a high-speed recording medium such as a hard disk, it can be provided as a resident area or a user area of the system.

FIG. 3 shows the interactive processing application 4
It is a schematic block diagram which comprises. Here, the concept of the interactive processing application 4 includes not only a program but also a data storage area.

In FIG. 3, an input / recognition unit 101 processes and analyzes voice data input from the input device 1 and performs voice recognition. As described above, well-known speech recognition methods such as the DP pattern matching method and the Hidden Markov method are employed for speech recognition. For example, when "I" is input, this is referred to as "wa" or "ta". Recognize for each phoneme, such as "shi". Further, based on these recognized phonemes, syntax analysis is performed to recognize the meaning. A so-called morphological analysis is adopted for the syntax analysis. For example, the above "wa""ta"
When the phoneme of "shi" is recognized, it is recognized as "me".

The dialog processing unit 102 includes an input / recognition unit 101
This is a part that determines the contents to be responded to the user based on the result from. Here, the contents are determined with reference to the output data table 103. Output data table 103
Is a part for storing output data from the apparatus for each state of conversation with the user. FIG. 4 shows a configuration example of the output data table. This table stores output data (audio output patterns) and types. The type indicates the state of conversation with the user. Output data is stored for each conversation situation, that is, for each type. Each output data includes
It includes <xxx> and <yyy> and the transformation attribute indicated by () following them. <Xxx> and <yyy>
Indicates a word to be transformed. <Xxx>
Is the part where the user input is returned as is,
<Yyy> is a portion into which the result dynamically generated by the system is inserted. In the example shown in FIG. 1, “Nagaoka Kyoto” is <xxx>, and “Komikyo” is <yyyy.
>. <Xxx> to the entire output data
And <yyy> portions are incorporated. The transformation attribute indicated by () will be described later.

The transformation section 104 for transforming the output content is a section for transforming the output data selected by the dialog processing section 102. The modification unit 104 refers to a character-based additional word table 105, a word-based additional word table 106, a modification counter 107, and a modification content table 108.

FIG. 5 shows a configuration example of the modification content table 108. This table defines the type of transformation (corresponding to the attribute) and the specific transformation content. For example, if the type is deformation 0, the deformation content is “no deformation”. That is, it does not deform. When the type is the deformation 1, the contents of the deformation are “loud volume” and “low speed”. That is, by increasing the volume output of the target <xxx> or <yyy> and lowering the audio output speed, control is performed so that the speaker speaks loudly and slowly. As shown in the figure, the modification includes a change in volume, a change in speed, the presence or absence of word repetition, the presence or absence of an additional word for explanation, and the like.

FIG. 6 shows a configuration example of the character-based additional word table 105.

This table is composed of one kanji character and an additional word for explaining the character. In this example,
As will be described later, this character-based additional word table is used to explain words that are not in the word-based additional word table and to describe each word by separating each character.

FIG. 7 shows an example of the structure of the additional word table 106 for each word. This table includes a character string of a word and an additional word for explaining the word.

FIG. 8 shows an example of the configuration of the modification counter 107. The counter includes a word and a count unit that stores the number of times the word has been deformed in the past. The value of the counter is used to determine whether to transform the specified word. The larger this value is, the higher the possibility of not performing the deformation is, and the smaller this value is, the higher the possibility of performing the deformation is.

Next, the general operation will be described with reference to FIG.

ST101 The transformation counter 107 is cleared to 0. The deformation counter 107 is used to determine whether to perform deformation by referring to the value. When the value is small, the transformation is performed, and when the value is large, the transformation is not performed. In step ST101, the counter is initialized to 0 so that the deformation is performed.

ST102 Speech recognition is performed from speech data input by the user, and morphological analysis is performed to obtain a word string. Also, the recognition certainty for each word is obtained. It should be noted that a technique for obtaining the certainty of the recognition when performing the voice recognition is general. For example, in the DP matching method, it is possible to numerically determine the recognition certainty by obtaining a correlation value with reference data (reference data).

ST103 The voice output unit determines whether or not the additional word is currently being output. The case where the input voice is recognized in ST102 while the additional word is being output is when there is a voice input expressing annoyance such as "I understand" from the user. Therefore, at this time, ST104
Then, the output of the additional word is interrupted, and the transformation counter 107 is set to infinity (ST105). ST103-10
The operation 5 is such that, when an interrupt input is made by the user during the output of the additional word, the user determines that the explanation of the additional word is bothersome and does not output the additional word. It is to be.

ST106 If the input voice recognized in ST102 is a voice expressing a confusing word such as "E" or "Nani" or a voice indicating a detailed request such as "Detailed", the process proceeds to ST107. Set counter 107 to 0
clear. This action indicates that the user has failed to recognize the audio output from the device or that a detailed description is required. Therefore, in this case, the transformation counter is cleared to 0, and the transformation is performed based on the transformation contents described in the output data table.

ST108 The data to be output by the system is selected from the output data table 103 based on the current dialogue situation. In general, the choice of output data is
It is selected in order from the smallest position in the table.
In the example shown in FIG. 4, output data of the "greeting" type is selected first, and then output data of the "confirmation" type is sequentially selected from the top. The method is not limited to the method of selecting the output data in order from the top, but it is also possible to dynamically select the output data according to the conversation situation at that time.

ST109-ST111 Here, the later-described transformation processing is performed on each of the transformation target words incorporated in the selected output data.

ST112 Speech-synthesize the transformed data and output.

ST113 It is determined whether or not the current output is a mere repetition of the immediately preceding output. If it is not a repetition, the transformation counter 107 is incremented for all the transformation target words incorporated in the output. This operation means that, as the number of times of deformation has been increased in the past, the user correctly understands the contents of the word and does not perform the deformation.

After the above operation is completed, the process returns to ST102 and waits for a new input.

FIG. 10 shows a control flow of the transformation processing unit in ST110.

ST201, ST202 The word to be transformed is input by the user (corresponding to <xxx>)
If the certainty degree of the voice recognition is lower than a predetermined value, the corresponding deformation attribute is attached to the word (S
T204). In the example shown in FIG. 1, when there is a user utterance of "Nagaokakyo" and the certainty of the voice recognition is low,
Proceeding to T204, a deformation attribute is added to the corresponding word. The deformation attribute at this time is “deformation 1” because it is a dialogue state of the “confirmation” type in FIG. In other words, in the actual deformation, “volume is high” and “speed is low” from FIG.

ST203: The part to be transformed is a part dynamically generated on the system side (corresponding to <yyy>),
If the transformation counter 107 does not exceed a certain value, the corresponding transformation attribute is added to the word (S
T204). In the example shown in FIG. 1, when the system outputs the message in a dialogue state of “Kyomyoji is recommended.” And “Simple answer”, if the deformation counter 107 does not exceed a certain value, “Deformation 2” is performed. Is added as an attribute. Therefore, "Komiji" (slowly and reliably output) "recommended" (fast and quick output) is displayed.

FIG. 11 is a control flow of the voice synthesizing section in ST112 of FIG.

ST301 If the contents of the modification include "with additional words", the operation proceeds to ST302. Here, it is determined whether or not the corresponding target word is included in the word-by-word additional word table 106. If the target word is included, a corresponding additional word is generated (ST303), and this additional word is added to the target word. Insert immediately before (ST304). Otherwise, for each of the characters contained in the word, an additional word for each character is obtained using the character-specific additional word table 105, and a particle "ni" is inserted between the additional words. One additional word string is generated (ST305). This additional word is inserted immediately after the target word (ST306). It is also possible to set a tone color change attribute for the additional word portion. In ST307, this process is performed, and the additional word and the corresponding target word or the like can be distinguished by hearing.

ST308 Performs acoustic processing according to the contents of the transformation to synthesize speech.

FIG. 12 shows an image diagram of a system using an audio output device according to another embodiment of the present invention.

This system is a voice input system for recognizing all sentences uttered by the user. For example, it is used when a user reads a manuscript written on paper and converts it into accurate digitized data. In this voice input system, in order to create correct digitized data, when there is a homonym or the like, dialog processing is performed with the user to prevent erroneous recognition. In the example shown in FIG.
An image of the scene where "Please watch out for that bridge" was first uttered. In this case, it is assumed that the speech recognition result of the system is “Take care of that end”. That is, the system incorrectly recognizes "bridge" as "edge". In this case, the system is a “bridge” and “edge”
Is added to distinguish between. That is, the system outputs a voice saying "Take care of the end of the tail." At this time, the user will know that the system is making a mistake.
Speak control words such as "repeat" and "correct" to enter the dialogue for correction. For example, when trying to make a correction, the user specifies a word using an additional word, such as "XX of correction XX". In the example shown in FIG. 12, “correction ○ bridge chopstick” is uttered. At this time, the system determines whether "bridge" is registered in the system as an additional word specifying "bridge". If it is registered, it performs a correction process and outputs the sound again. That is,
"Please watch out for the bridge of that bridge." If “bridge” is not registered in the system as an additional word specifying “bridge”, possible recognition candidates are sequentially added with additional words and output. If the user outputs an incorrect candidate,
Say a word that expresses the negation of "No." If the system recognizes no, it outputs the next recognition candidate. By this repetition, when the word desired by the user is output as a voice, the user has an opportunity to say a positive expression of “yes”. Then, the system corrects the recognition result and additionally updates the system with the additional word “bridge” used by the user. Thereafter, "bridge" can be used as an additional word to specify "bridge".

In the example shown in FIG. 12, after the user utters “Take care of that bridge”, the user knows that the system is giving an incorrect recognition result, and Uttered, the system recognized it and uttered the same content again, and the user uttered "Correction bridge chopsticks", and the system output "Chopsticks eating rice" as the first candidate,・ The user utters “No”, the system outputs “Which bridge is over” as the next candidate, the user utters “Yes”, and the system “corrected. Watch out for this bridge. Please output. "

The configuration diagram of the above system is the same as the configuration shown in FIG. The main control flow is the same as the flow shown in FIG. FIG. 13 shows that the user can select “Correction XX
This is a control flow in the case where the correction process is performed by uttering "○ of ○○".

ST601 Here, it is determined whether or not the result of recognition of the input speech (ST102 in FIG. 9) is equivalent to a correction command. That is, it is determined whether or not the user has uttered “XX of correction XX”. In the case of a correction command, the processing after ST602 is processed.

ST602 From the timing of the correction command uttered by the user, the user specifies which word in the output data of the system is to be used for correction. In the example shown in FIG. 2, the system utters “correction” immediately after the system recognizes the utterance of “repetition” of the user and outputs the voice of “that shit”. "Is specified as a word to be corrected by the word immediately after the voice output.

ST603 Extract the post-correction and additional words contained in the correction command. In the example shown in FIG. 12, the corrected word is "Hashi" and the additional word is "Bridge".

ST604 It is determined whether or not the corrected word and the additional word have already been registered in the character-based additional word table 105 or the word-based additional word table 106 of the system. If registered, the process proceeds to ST605 to select a word corresponding to the additional word. If not, S
At T606, recognition candidate words such as homonyms are sequentially output together with the registered additional words, and the user is asked for confirmation.

ST607 Here, it is determined whether or not there is a word desired by the user in the output of the sequentially output candidates. This determination is made based on whether or not the user has uttered a negative or positive expression such as “No” or “Yes”. If there is a word requested by the user, S
At T608, the additional word is newly registered in the word-specific additional word table or the character-specific additional word table. Otherwise, the fact that the word is not found is output and the user is prompted to re-input the original sentence.

[0077] ST609 Then, the recognition content is corrected,
The correction result is output as audio (ST601), and the process returns to the original position.

FIG. 14 is an image diagram of a system using an audio output device according to another embodiment of the present invention.

This system is obtained by attaching a camera to the apparatus shown in FIG. This camera performs confusing detection of the user. In the example shown in FIG. 1, when the user does not understand the contents of the audio output of the system,
Is expressed by making an utterance such as, for example, when the user feels confused, for example, there is a custom or a habit of expanding both hands. In addition, a rule may be determined in advance. In such a case, it is possible to detect that the user is in a troubled state by detecting the spread state of the hand based on the image.
Therefore, in this case, it is possible for the user to inform the system of the embarrassment only by spreading his hand without saying "eh" or "what".

FIG. 15 is an image diagram of a system using an audio output device according to another embodiment of the present invention.

In this system, in addition to the operation of transforming the output data, the lamp is turned on or blinked according to the output timing when the word to be incorporated is output. In the example shown in FIG. 15, when the certainty degree of the voice recognition is equal to or less than a certain value, the lamp is turned on and off at the same time as outputting the target word (where the voice recognition is not confident). Also, the lamp is blinked for a part for outputting important information dynamically generated on the system side. Therefore, FIG.
In the image diagram shown in FIG. 5, “Nagaoka Kyoji” and “Kogami Kyoji” are sound-output while blinking the lamp.

The above-mentioned blinking operation of the lamp may be performed independently without performing the voice transformation processing on those words.

FIG. 16 shows an image diagram of a system using an audio output device according to another embodiment of the present invention.

In this system, a vibrator is used instead of the lamp shown in FIG. That is, the vibrator is vibrated instead of blinking the lamp. In this case, the user has the vibrator.

In addition, a small output buzzer can be used in place of the lamp or vibrator.

[0086]

According to the first aspect of the present invention, when a user outputs a voice that is confusing because the user cannot recognize the voice output from the device, the device detects the voice and makes it easy for the user to understand. Is transformed and output again, so that the recognition of the user during the voice dialogue is assured, and a smooth dialogue can be performed.

According to the second aspect of the present invention, when a similar sound output is repeatedly performed, the user feels annoyed and only issues an interrupt sound, and does not repeat thereafter. , And a smooth dialogue becomes possible.

According to the third aspect of the present invention, since the user's troublesome state is detected by the imaging means such as a camera, when the troublesome state cannot be detected by voice recognition, for example, even when the surrounding noise environment is severe, etc. The smoothness of the dialogue is improved.

According to the fourth aspect of the present invention, the reliability of the dialog can be improved even when the noise environment around the user is severe.

According to the fifth aspect of the present invention, even when a homonym or an esoteric term is included in the output data, the user can easily recognize the meaning and content.

According to the sixth aspect of the present invention, even when it is not possible to add an additional word to a word, by adding an additional word to each character, the user can obtain a homonymous word or an esoteric word. You can easily understand the meaning.

According to the seventh aspect of the present invention, in a system in which a whole sentence uttered by a user is voice-recognized and converted into digitized data, even if a homonymous word or an esoteric word exists, an additional word is added. By confirming and outputting, incorrect electronic data can be prevented from being created.

According to the invention of claim 8, even when the additional word is not registered on the device side, candidates are sequentially output, and the user who outputs the semantically correct additional word is sent from the user. Since the additional word can be registered as a new additional word based on the voice output indicating affirmation of the above, the additional word table can be learned and updated, so that a more accurate and smooth dialogue according to the number of uses becomes possible. come.

According to the ninth aspect of the present invention, the notification by the voice-out means is performed at the same time as the word deformation, so that the user's recognition becomes more reliable.

According to the tenth aspect of the present invention, since no deforming means is required, the entire apparatus can be simplified.

[Brief description of the drawings]

FIG. 1 is an image diagram of a system to which an audio output device according to an embodiment of the present invention is applied;

FIG. 2 is a diagram showing a system configuration;

FIG. 3 is a schematic block diagram of the system.

FIG. 4 is a configuration example of an output data table;

FIG. 5 is a configuration example of a modification content table.

FIG. 6 is an example of an additional word configuration for each character.

FIG. 7 is an example of an additional word configuration for each word.

FIG. 8 is a configuration example of a deformation counter.

FIG. 9 is a diagram showing a schematic flow.

FIG. 10 is a diagram showing a control flow of a transformation processing unit.

FIG. 11 is a diagram showing a control flow of a speech synthesis unit.

FIG. 12 is an image diagram of a system using an audio output device of another embodiment.

FIG. 13 is a diagram showing a control flow of a correction processing unit.

FIG. 14 is an image diagram of a system using an audio output device of another embodiment.

FIG. 15 is an image diagram of a system using still another audio output device.

FIG. 16 is an image diagram of a system using still another audio output device.

Claims (13)

[Claims] 1. A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from input voice, an output data table for storing output data to be output as voice from the apparatus for each conversation state with a user, and a voice recognition result. Judge the dialogue status from
Output data selection means for selecting output data to be output at that time from the output data table, an audio output unit for outputting the selected output data as an audio signal, and an audio signal representing the user's confusing sound with respect to the audio output from the apparatus. A voice output device comprising: means for detecting from a recognition result; and deforming means for deforming a word incorporated in output data when detecting a voice indicating the confusion and causing the voice output unit to re-output the word. 2. A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from an input voice, an output data table for storing output data to be output as voice from the apparatus for each conversation state with a user, and a voice recognition result. Judge the dialogue status from
Output data selecting means for selecting output data to be output at that time from the output data table, a voice output unit for outputting the selected output data as voice, and voice recognition of a user interrupt voice for voice output from the device. A sound output device comprising: means for detecting from a result; and deforming means for deforming a word incorporated in output data when the interrupt sound is detected and outputting the word again at the sound output unit. 3. A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each dialogue state with a user, and a voice recognition result. Judge the dialogue status from
Output data selection means for selecting output data to be output at that time from the output data table, an audio output unit for outputting the selected output data as an audio, an imaging means for imaging the user, and an audio output from the apparatus. Means for detecting a confusing state of the user from the image of the user taken by the imaging means, and a word incorporated in the output data when the confusing state is detected from the image of the user, and transformed by the audio output unit. An audio output device comprising: a transforming means for re-outputting. 4. A voice input unit for inputting voice, a voice recognition unit for performing voice recognition from the input voice, an output data table for storing output data to be output as voice from the apparatus for each dialogue state with a user, and a voice recognition result. Judge the dialogue status from
Output data selection means for selecting output data to be output at that time from the output data table, a voice output unit for outputting the selected output data as a voice, and detecting the recognition certainty of the input voice. Deforming means for deforming a word incorporated in the output data when the value is equal to or less than a threshold value and outputting the word again at the voice output unit. 5. A word-by-word additional word table for storing an additional word for explaining a word incorporated in output data, wherein the deforming unit deforms by outputting a voice in which the additional word is added to the word. An audio output device according to any one of claims 1 to 4. 6. An additional word table for storing an additional word for each character for explaining each character of a word incorporated in the output data, wherein the deforming means adds an additional word to each character constituting the word. The audio output device according to claim 1, wherein the audio output device is deformed by outputting audio. 7. A voice input unit to which a voice is input, a voice recognition unit that performs voice recognition from the input voice, and a unit that generates a voice recognition result as output data in order to repeat the input voice on the device side, An audio output unit that outputs output data as audio, a word-specific additional word table that stores additional words for explaining words to be incorporated in the output data, and transforms the incorporated words into words obtained by adding additional words to the words. And a deforming means for re-outputting by the output means. 8. When a word to which an additional word is added is spoken as a correction word by a user after the output data in which the transformed word is incorporated, the corresponding word is input as the corrected word. Means for converting to a word-by-word additional word table, when the additional word input by the user is unregistered, the homonymous word and the additional word are sequentially output as candidates as voice, and the user adds the correct additional word. 8. The audio output device according to claim 7, further comprising: a table updating unit that, when there is an input indicating that the word is present, adds a corresponding word as a candidate additional word at that time to the additional word table for each word. 9. When the word incorporated in the output data is transformed by the transformation means and output by the audio output unit, a notification is issued by an external means such as a lamp, a buzzer, or a vibrator when the incorporated word is output. The audio output device according to any one of claims 1 to 9, comprising means.
8. The audio output device according to claim 7, comprising: a new unit. 10. The voice according to claim 1, further comprising a notification unit that performs notification by an external unit such as a lamp, a buzzer, or a vibrator when outputting the incorporated word, instead of the deformation unit. Output device. 11. A voice input step, a voice recognition step for performing voice recognition from an input voice, and an output data table for storing output data from the apparatus for each state of interaction with a user, and referring to a voice recognition result. An output data selection step of determining an interaction situation and selecting output data to be output at that time from the output data table; an output step of outputting the selected output data by voice; Recording a voice input program comprising: a step of detecting a voice representing a voice from a voice recognition result; and a deforming step of deforming a word incorporated in output data according to the confusing state and re-outputting the word at the output step. Audio output program recording medium. 12. A speech input step, a speech recognition step of performing speech recognition from input speech, and an output data table for storing output data from the apparatus for each conversation status with a user, and performing a dialog based on a speech recognition result. Judge the situation,
An output data selecting step of selecting output data to be output at that time from the output data table; an output step of outputting the selected output data by voice; and a voice recognition result of a user's interrupt voice for voice output from the device. A sound output program recording medium, comprising: a step of detecting a sound to be incorporated into output data; and a step of deforming the sound incorporated in the output data in accordance with the interrupt sound and outputting the sound again at the output step. 13. A voice input step, a voice recognition step of performing voice recognition from input voice, and an output data table storing output data from the apparatus for each dialogue state with a user, and based on the voice recognition result, a dialogue state. The output data selection step of selecting output data to be output at that time from the output data table, the output step of outputting the selected output data, and the confusing state of the user with respect to the audio output from the device. A sound output program comprising: a step of detecting a user from an image obtained by imaging the user; and a deforming step of deforming a sound incorporated in the output data in accordance with the confusing state and re-outputting the sound in the output step. recoding media.