JP3838159B2 - Speech recognition dialogue apparatus and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a speech recognition dialogue apparatus that outputs a response to a content spoken by a speaker, and particularly, when there are a plurality of speakers around the voice recognition dialogue apparatus, the dialogue is concentrated only on a specific speaker. The present invention relates to a speech recognition dialogue apparatus that can perform a dialogue or an alternative dialogue with a plurality of speakers.
[0002]
[Prior art]
In a speech recognition dialogue apparatus that outputs a response to a speaker's utterance by voice, it is necessary to recognize the utterance of the speaker at a high recognition rate. In order to increase the recognition rate, a speech recognition device has been conventionally proposed in which the influence of ambient noise and the like is reduced, and speech from a specific speaker is captured with good quality (for example, specially No. 2000-148184).
[0003]
FIG. 7 is a block diagram showing a configuration of a speech recognition apparatus described in Japanese Patent Laid-Open No. 2000-148184. Referring to FIG. 7, a voice information input unit 70 configured to vary directivity characteristics, sensitivity characteristics, and the like of a microphone array, a voice input control unit 71 that adjusts directivity characteristics, sensitivity characteristics, and the like of the voice information input unit 70. A voice feature vector extraction unit 72 that performs A / D conversion on the voice signal input from the voice information input unit 70 based on the control of the voice input control unit 71, performs frequency analysis, and converts it into a voice feature vector sequence; A speech recognition unit 73 that performs speech recognition based on the speech feature vector obtained from the speech feature vector extraction unit 72, a recognition result display unit 74 that displays a recognition result of the speech recognition unit 73, and an imaging device such as a camera. The image information input part 75 and the image information analysis part 76 which analyzes the image information input from the image information input part 75 are provided.
[0004]
Subsequently, the operation of the speech recognition apparatus described in Japanese Patent Application Laid-Open No. 2000-148184 will be described. In FIG. 7, the image information analysis unit 76 analyzes the image data obtained from the image information input unit 75 and detects the position of the speaker in the image. The position of the speaker in the image can be obtained by extracting the face image of the speaker and tracking it. The voice input control unit 71 controls the directivity characteristics, input characteristics, and direction of the voice information input unit 70 based on the speaker position data transmitted from the image information analysis unit 76.
[0005]
[Problems to be solved by the invention]
However, when the above-described conventional speech recognition apparatus is used for a speech recognition dialogue apparatus, the following problems occur.
[0006]
The first problem is that, while a plurality of speakers are around the speech recognition dialogue apparatus, it is not possible to carry out an alternative dialogue with a plurality of speakers in different directions.
[0007]
The reason is that in order to improve the speech recognition rate of a specific speaker, the sensitivity characteristics of the microphone and the directivity characteristics of the microphone are adjusted in the direction of the specific speaker, and the speaker in the other direction is adjusted. This is because it is difficult to capture the sound.
[0008]
The second problem is that, while a plurality of speakers are around the speech recognition dialogue apparatus, it is not possible to conduct a concentrated dialogue only with a specific speaker in the same direction.
[0009]
The reason is that the microphone sensitivity characteristics and microphone directivity characteristics are only adjusted in the direction of the speaker, so that voices spoken by other speakers from the same direction are also captured and recognized. .
[0010]
OBJECT OF THE INVENTION
It is an object of the present invention to have a plurality of speakers around a speech recognition dialogue apparatus, sometimes to concentrate on a conversation with a specific speaker, or sometimes to have an alternate dialogue with a plurality of speakers. The object of the present invention is to provide a speech recognition dialogue apparatus that can be switched naturally during dialogue.
[0011]
[Means for Solving the Problems]
The speech recognition dialogue apparatus according to the present invention manages a speaker to manage and control the degree of concentration on a speaker based on speaker position specifying information obtained by analyzing spoken speech information, collation speaker information, and a speech recognition result. 1 and 14 (FIG. 1) and the speaker concentration control unit (14 in FIG. 1) stores information necessary for determining the concentration, and is referred to and updated by the speaker. And a concentration management database (20 in FIG. 1).
[0012]
More specifically, the speech recognition dialogue apparatus of the present invention is
A voice input unit (10 in FIG. 1) for capturing voice information;
A speaker position specifying unit (11 in FIG. 1) for specifying the direction of the speaker who has spoken;
A speaker verification unit (12 in FIG. 1) for identifying a speaker who has spoken,
A voice recognition unit (13 in FIG. 1) that analyzes voice information input from the voice input unit (10 in FIG. 1) and recognizes a voice;
A degree-of-concentration setting table (24 in FIG. 1) in which a specific speaker identification name indicating a specific speaker and a concentration level for the specific speaker indicated by the specific speaker identification name are set;
It is determined whether or not to enable the speaker's speech based on the content of the concentration level setting table (24 in FIG. 1) and the speaker specified by the speaker verification unit (12 in FIG. 1). If determined to be valid, the level of concentration determined based on the recognition result of the voice recognition unit (13 in FIG. 1) for the utterance of the speaker and the speaker verification unit (12 in FIG. 1) The level and the specific speaker identification name in the concentration setting table (24 in FIG. 1) are updated using the identified speaker identification name, and the updated concentration setting table (24 in FIG. 1). Concentration control unit for controlling the directivity and direction of the voice input unit (10 in FIG. 1) based on the content of the speaker and the direction of the speaker specified by the speaker position specifying unit (11 in FIG. 1) (14 in FIG. 1),
And a voice output unit (19 in FIG. 1) that outputs a response to the recognition result of the utterance determined to be valid by the concentration control unit (14 in FIG. 1).
[0013]
Furthermore, the voice recognition dialogue apparatus of the present invention can change the concentration level of a specific speaker set in the concentration level setting table (24 in FIG. 1) when a predetermined event occurs.
Another event management unit (16 in FIG. 1) for detecting that a predetermined event has occurred, and
The concentration level control unit (14 in FIG. 1) is set in the concentration level setting table (24 in FIG. 1) when the occurrence of the predetermined event is detected by the other event management unit (16 in FIG. 1). It has a configuration for changing the level of concentration.
[0014]
More specifically,
The predetermined event is that there is no utterance for a predetermined time by a specific speaker whose specific speaker identification name is set in the concentration setting table (24 in FIG. 1); and
The concentration control unit (14 in FIG. 1) detects the occurrence of the predetermined event in the other event management unit (16 in FIG. 1), and sets it in the concentration setting table (24 in FIG. 1). When the level of concentration being made is high enough to enable only the utterance of the specific speaker indicated by the specific speaker identification name set in the concentration setting table (24 in FIG. 1), The concentration level set in the concentration level setting table (24 in FIG. 1) is lowered to a level that also enables speech by other speakers.
[0015]
[Action]
When talking with multiple speakers, control the concentration level on the speaker based on the preset concentration level change condition, and input a voice to a microphone array, etc., according to the concentration level. The directivity and direction of the unit (10 in FIG. 1) are adjusted. Further, the utterances of speakers other than the specific speaker are invalidated according to the level of concentration.
[0016]
If the concentration level for a specific speaker is determined based only on the utterance content of the specific speaker, if the specific speaker leaves the speech recognition dialogue device after making a speech that increases the concentration level, The state in which the speaker's speech is invalidated continues, and other speakers cannot interact with the speech recognition dialogue apparatus. Therefore, when the occurrence of a predetermined event (for example, a time during which there is no utterance by a specific speaker continues for a predetermined time) is detected in another event management unit (16 in FIG. 1), a concentration level control unit (14 in FIG. 1) However, the level of the concentration set in the concentration setting table (24 in FIG. 1) is lowered to a level at which utterances by other speakers are also effective. As a result, other speakers can also interact with the speech recognition dialogue apparatus.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described in detail with reference to the drawings. Referring to FIG. 1, a first embodiment of a speech recognition dialogue apparatus according to the present invention includes a speech input unit 10, a speaker position specifying unit 11, a speaker verification unit 12, a speech recognition unit 13, Concentration control unit 14, speech input control unit 15, other event management unit 16, dialogue control unit 17, speech synthesis unit 18, speech output unit 19, and database 20 for managing speakers and concentration level It consists of and.
[0018]
The voice input unit 10 has a function of converting voice information into an electrical signal. In addition, the voice input unit 10 can change directivity and direction, and is configured by, for example, a microphone array in which a plurality of microphones are arranged in a circular shape at regular intervals.
[0019]
The speaker position specifying unit 11 has a function of analyzing the voice information input from the voice input unit 10 and specifying the direction of the speaker. For example, when the voice input unit 10 is configured by a microphone array in which a plurality of microphones are arranged in a circular shape, the direction of the microphone with the highest output level is set as the speaker direction. The direction of the microphone is a direction with respect to a reference direction of the speech recognition dialogue apparatus, and an angle between a reference microphone of a plurality of microphones and a microphone having the highest output level, and an angle between the reference direction and the reference microphone are determined. It is obtained by adding.
[0020]
The speaker verification unit 12 has a function of analyzing voice information input from the voice input unit 10 and checking the voice information of a registered speaker to identify the speaker.
[0021]
The voice recognition unit 13 has a function of recognizing voice by analyzing voice information input from the voice input unit 10.
[0022]
The concentration level control unit 14 includes speaker position specifying information input from the speaker position specifying unit 11, verification speaker information input from the speaker verification unit 12, speech recognition results input from the speech recognition unit 13, and others. It has a function of controlling the degree of concentration on the speaker based on the notification from the event management unit 16.
[0023]
More specifically, the concentration level control unit 14 has the following functions.
[0024]
Whether or not to enable the utterance of the speaker specified by the verification speaker information based on the content of the concentration level setting table 24 and the verification speaker information (speaker identification name) from the speaker verification unit 12 The function to determine whether.
A function of rejecting the recognition result input from the voice recognition unit 13 when it is determined not to be valid.
A function of passing the recognition result to the dialogue control unit 17 to the voice recognition unit 13 when it is determined to be valid.
If it is determined to be valid, the concentration level is determined based on the recognition result of the voice recognition unit 13 and the content of the transition condition table 21, and the determined concentration level and the speaker verification unit 12 A function of updating the content of the concentration level setting table 24 based on the verification speaker information.
The direction and directivity of the voice input unit 10 are adjusted with respect to the voice input control unit 15 based on the contents of the updated concentration degree setting table 24, the contents of the definition table 22, and the information table 23. Function to direct.
[0025]
In addition, each table 21-24 in the database 20 is demonstrated in detail later.
[0026]
The voice input control unit 15 has a function of adjusting the directivity and direction of the microphone array or the like of the voice input unit 10 (the direction of the reference microphone with respect to the reference direction of the voice recognition dialogue apparatus) according to an instruction from the concentration control unit 14. .
[0027]
The other event management unit 16 has a function of managing other events such as time other than voice input, and notifying the concentration degree control unit 14 of the occurrence of the event.
[0028]
The dialogue control unit 17 has a function of managing dialogue contents based on the voice recognition result and speaker verification information sent from the concentration degree control unit 14 and determining the next response content.
[0029]
The speech synthesizer 18 has a function of generating synthesized speech of response contents input from the dialogue control unit 17.
[0030]
The voice output unit 19 has a function of outputting the synthesized voice input from the voice synthesis unit 18 and is configured by a speaker or the like.
[0031]
The database 20 includes a transition condition table 21, a definition table 22, an information table 23, and a concentration level setting table 24 that are used when the concentration level control unit 14 controls the concentration level on a speaker.
[0032]
The change condition table 21 stores various conditions for changing the level of concentration with respect to a specific speaker. Each condition includes a condition content, a current concentration level (current level), and a concentration level to be changed (transition level). For example, the condition No. 1 indicates that the level is changed to “low” when “thank you” or “you are better” is spoken when the current concentration level is “medium”. Further, for example, Condition No. 7 indicates that the level is changed to “medium” when there is no utterance by a specific speaker for 30 seconds when the current concentration level is “high”.
[0033]
In the definition table 22, the contents of control performed by the concentration level control unit 14 are defined for each level of concentration level. For example, when the concentration level is “low”, the concentration control unit 14 sets the directivity of the voice input unit 10 to −180 degrees to 180 degrees, and the specific talk set in the concentration setting table 24. Voice recognition results other than those who are not active are also enabled. When the concentration level is “high”, the concentration level control unit 14 sets the direction of the voice input unit 10 to the direction of the specific speaker, sets the directivity to −45 degrees to 45 degrees, and sets the concentration level setting table 24. Disable speech recognition results for non-specific speakers set to.
[0034]
In the information table 23, the speaker identification name specified by the speaker verification unit 12 and the direction specified by the speaker position specifying unit 11 are registered in association with each other. The example of FIG. 1 indicates that the father is at 0 degrees, the mother is at 90 degrees, and the unknown person is at 180 degrees with respect to the reference direction of the speech recognition dialogue apparatus.
[0035]
In the concentration level setting table 24, the level of the current concentration level and the identification name of the specific speaker as the target are set correspondingly. The example of FIG. 1 shows that the current concentration level is “high” and the father is the target.
[0036]
Next, the operation of the present embodiment will be described in detail with reference to FIG. 1, FIG. 2, and FIG.
[0037]
First, the operation when the speaker speaks will be described with reference to FIGS. When the speaker speaks, the voice information input via the voice input unit 10 such as a microphone array is output to the speaker position specifying unit 11, the speaker verification unit 12, and the voice recognition unit 13, respectively. The speaker position specifying unit 11 analyzes the input voice information, specifies the direction of the sound source of the speaker, and outputs the speaker position specifying information to the concentration level control unit 14. The speaker verification unit 12 analyzes the input voice information, compares it with the registered speaker's voice information, identifies the speaker, and outputs the verification speaker information to the concentration level control unit 14. The voice recognition unit 13 analyzes the input voice information and outputs the voice recognition result to the concentration degree control unit 14.
[0038]
The concentration control unit 14 updates the position direction of the collation speaker information table 23 based on the input speaker position specifying information and collation speaker information (FIG. 2, S20).
[0039]
Next, it is determined whether or not the concentration level set in the concentration level setting table 24 is a concentrated conversation state (S21). As a result of the determination, if the level is “high” indicating the concentrated conversation state, it is determined whether or not the collated speaker matches the identification name of the specific speaker in the concentration level setting table 24 (S22). .
[0040]
If they do not match, the input speech recognition result is rejected (S23). On the other hand, if they match, the transition condition table 21 is searched to search for a condition where the current level matches the level set in the concentration setting table 24 and the condition content matches the voice recognition result ( S24). Even when it is determined in step S21 that the level is not “high”, the process of step S24 is performed.
[0041]
If the corresponding condition cannot be found in step S24, the process of step S26 is performed. On the other hand, if the corresponding condition can be found, after changing the concentration level set in the concentration setting table 24 to the transition level in the condition found in step S24 (S25), The process of step S26 is performed. In step S <b> 26, a process of changing the identification name of the specific speaker set in the concentration level setting table 24 to the identification name of the speaker specified by the speaker verification unit 12 is performed.
[0042]
Next, the speaker concentration control unit 14 refers to the concentration setting table 24 and the information table 23, and uses the position and direction of the specific speaker as direction setting information such as a microphone array to the voice input control unit 15. In addition to output, the definition table 22 is referred to, and directivity setting information such as a microphone array defined corresponding to the current level of concentration is output to the voice input control unit 15 (S27). The voice recognition result and the verification speaker information are output to the dialogue control unit 17 (S28).
[0043]
The voice input control unit 15 determines the finger characteristics and direction of the microphone array of the voice input unit 10 based on the direction and directivity setting information of the microphone array and the like input from the speaker concentration control unit 14. adjust.
[0044]
The dialogue control unit 17 determines the next response content based on the speech recognition result and the collation speaker information input from the speaker concentration control unit 14, and outputs the response content to the speech synthesis unit 18. To do.
[0045]
The voice synthesizer 18 generates a synthesized voice from the input response content, and outputs the synthesized voice via the voice output unit 19 such as a speaker.
[0046]
Next, the operation when the other event management unit 16 detects the occurrence of a predetermined event will be described with reference to FIGS. 1 and 3. When detecting the occurrence of a predetermined event, the other event management unit 16 notifies the concentration level control unit 14 of the type of event that has occurred.
[0047]
As a result, the concentration level control unit 14 searches the transition condition table 21, and the condition where the current level matches the level set in the concentration level setting table 24 and the condition content matches the type of the notified event. Is searched (FIG. 3, S31).
[0048]
If the corresponding condition cannot be found in step S31, the concentration level control unit 14 ends the process. On the other hand, when the corresponding condition can be found, the concentration control unit 14 changes the concentration level set in the concentration setting table 24 to the transition level in the searched condition. (S32), with reference to the definition table 22, directivity setting information such as a microphone array defined corresponding to the current level of concentration is output to the voice input control unit 15 (S33). Processing ends.
[0049]
Next, the contents of the transition condition table 21 and the definition table 22 in the database 20 are as shown in FIG. 1, and the concentration level that can capture utterances from all directions as the concentration level is diverged in the concentration level setting table 24. The operation of this exemplary embodiment will be described in detail by taking as an example the case where “low” representing the state is set.
[0050]
For example, the back of the speech recognition dialogue apparatus, father respectively to the side, in a situation where a plurality of speakers, such as there are mothers in a different direction, father and utters "Hello".
[0051]
In this case, the concentration level control unit 14 first determines the information table 23 based on the speaker position specifying information input from the speaker position specifying unit 11 and the verification speaker information input from the speaker verification unit 12. The position direction of the father inside is updated (FIG. 2, S20). Thereafter, the concentration level control unit 14 changes the concentration level of the concentration level setting table 24 to “medium” according to the condition No. 5 in the transition condition table 21, and further selects a specific speaker to be the target of the concentration level as “ Change to "Father" (S21 is No, S24 is Yes, S25, S26). Thereafter, the concentration control unit 14 directs the direction of the voice input unit 10 to the back direction where the father who is the specific speaker is present and directivity according to the definition content of the concentration level “medium” in the definition table 22. The angle is adjusted to 90 to 90 degrees (S27). Furthermore, the degree of concentration control unit 14 performs the processing in step S28, thereby, the response to his father utters "Hello" is output from the audio output unit 19.
[0052]
After that, when the mother on the side makes an utterance that does not match the condition contents of the transition condition table 21 with “How are you?”, The concentration control unit 14 updates the position direction of the mother in the information table 23 (S20). Furthermore, while maintaining the normal conversation state of the level “medium” in the concentration level setting table 24, the specific speaker targeted for the concentration level is changed to “mother” (S21 is No, S24 is No). , S26). Thereafter, the concentration degree control unit 14 directs the direction of the voice input unit 10 toward the side surface where the mother who is the specific speaker is present, and adjusts the directivity to -90 degrees to 90 degrees (S27). Furthermore, the concentration control unit 14 performs the process of step S <b> 28, whereby a response to “how are you?” Spoken by the mother is output from the voice output unit 19.
[0053]
Thereafter, when the father speaks “I'm fine” or the like, the concentration control unit 14 updates the position direction of the father in the information table 23 in step S20, and the concentration in the concentration setting table 24 in step S26. In step S27, the direction of the voice input unit 10 is changed to a position where the father who is the specific speaker is located. In this way, it is possible to perform a dialogue with a voice recognition dialogue apparatus in which a father and a mother in different directions take the place.
[0054]
In such a normal dialogue state, when the father wants to talk in a state where the speech recognition dialogue device is concentrated on himself / herself, he / she utters “Please listen carefully”. Thereby, the concentration control unit 14 changes the position of the father in the information table 23 in step S20, and in step S25, the concentration level of the concentration setting table 24 is set in accordance with the condition No. 4 of the transition condition table 21. In step S26, the specific speaker to be focused is changed to “father”. In step S27, the direction of the voice input unit 10 is changed to the position direction of the father who is the specific speaker. And adjust the directivity to -45 degrees to 45 degrees. Under this circumstance, if the father continues the conversation, the voice input unit 10 also reduces the probability that the utterance and noise of the unrelated person will be captured, and the father's voice can be easily captured, thereby improving the voice recognition rate. For this reason, even if the mother speaks something in this situation, the probability that the voice input unit 10 captures the voice is reduced by the directivity adjustment result. Even if the voice input unit 10 captures the voice, the speaker verified by the speaker verification unit 12 is a mother and does not match the father of the specific speaker that is the target of concentration in the current concentration setting table 24. Therefore (S22 is No), the speech recognition result of the mother's speech content is rejected (S23).
[0055]
Next, in this situation, if the father makes an utterance that does not match the condition content of the transition condition table 21 such as “but yesterday”, the determination result in step S24 is No, so the concentration level setting is performed. The level of concentration on the table 24 remains “high”, and the concentrated conversation state with the father is maintained.
[0056]
Next, under this circumstance, if the father wants to stop the concentrated conversation, he speaks "I'm fine". Thereby, the concentration control unit 14 changes the concentration level of the concentration setting table 24 to “low” in step S25 in accordance with the condition No. 2 in the change condition table 21. The directivity is adjusted to -180 degrees to 180 degrees based on the definition content of the concentration level "low". In step S25, since the level in the concentration level setting table 24 is changed to “low”, the speech recognition result for other than the specific speaker is valid from the next time without being rejected (No in S21).
[0057]
Also, suppose that the father who is the specific speaker who is the target of concentration in the current concentration setting table 24 leaves the place with the concentration level set to “high”, that is, in a concentrated conversation state. Even in this case, based on the notification from the other event management unit 16, the mother and other speakers can interact with the speech recognition dialogue apparatus.
[0058]
That is, the other event management unit 16 notifies the concentration level control unit 14 of the type of the event when detecting an event in which the specific speaker's utterance time set in the concentration level setting table 24 continues for 30 seconds. . As a result, the concentration control unit 14 changes the concentration level in the concentration setting table 24 to “medium” based on the condition No. 7 in the transition condition table 21 (FIG. 3, S31 is Yes, S32). Thereafter, based on the directivity of the level “medium” in the definition table 22, the voice input control unit 15 is instructed to adjust the directivity of the voice input unit 10 to −90 degrees to 90 degrees (S33). ).
[0059]
Furthermore, if the time when there is no utterance by the specific speaker registered in the concentration level setting table 24 continues for 30 seconds, the other event management unit 16 notifies the concentration level control unit 14 of the event type again. Thereby, the concentration control unit 14 sets the concentration level in the concentration setting table 24 to “low” based on the condition No. 6 in the transition condition table 21 (S31 is Yes, S32), and then the definition table. 22 is instructed to adjust the directivity of the voice input unit 10 to −180 degrees to 180 degrees based on the directivity of level “low” in FIG. 22 (S33). As described above, if the duration of no speech continues for 30 seconds, the level in the concentration setting table 24 is changed from “high” to “medium” or from “medium” to “low”. Even if the father leaves the place with the level of concentration being “high”, the mother and other speakers can interact with the speech recognition dialogue apparatus.
[0060]
Note that the other event management unit 16 detects, for example, as follows, that a time period during which no utterance is made by a specific speaker registered in the concentration setting table 24 has continued for 30 seconds.
[0061]
A clear signal and a count start signal are input from the concentration level control unit 14 to the other event management unit 16. The clear signal is a signal that is output when the concentration level control unit 14 detects the start of speech of a specific speaker set in the concentration level setting table 24, and the count start signal is concentrated by the concentration level control unit 14. This signal is output when the end of the utterance of a specific speaker set in the degree setting table 24 is detected. The other event management unit 16 has a counter therein, and when a clear signal is input, the count value of the counter is set to “0”, the count operation is stopped, and a count start signal is input. The count operation is started. Then, when the count value becomes a value corresponding to 30 seconds, the concentration control unit 14 is notified that the non-speech time has continued for 30 seconds, and the count value is set to “0” and the count operation is resumed. To do.
[0062]
Then, for example, in a situation where a plurality of speakers like being father and mother to the back of the speech recognition dialogue system are in the same direction, for explaining the operation when the father utters "Hello". The contents of the transition condition table 21 and the definition table 22 are as shown in FIG. 1, and the concentration setting table 24 represents a state in which the concentration that can capture utterances from all directions is scattered as the concentration level. Assume that “low” is set.
[0063]
If father utters "Hello", the degree of concentration control unit 14, in step S20, updates the positional direction of the father in information table 23, in step S25, according to the conditions No5 of transition condition table 21, the concentration level setting table 24, the level of concentration is changed to “medium”, and “father” is set as a specific speaker to be the target of concentration in the concentration setting table 24 in step S26. Thereafter, in step S27, the concentration level control unit 14 adjusts the direction of the voice input unit 10 to the back direction where the father who is the specific speaker is present, based on the definition content of the concentration level “medium” in the definition table 22. In addition, the directivity is adjusted to -90 degrees to 90 degrees.
[0064]
In this situation, when the mother in the same direction speaks “How are you?”, The concentration level control unit 14 keeps the concentration level of the concentration level setting table 24 in the normal dialogue state of “medium” level. The specific speaker that is the target of the concentration degree is changed to the mother (S24 is No, S26). Since the level of the concentration level setting table 24 remains “medium”, the voice input unit 10 remains facing the same direction. In this situation, when the father speaks “I'm fine”, etc., the father who is in the same direction as the specific speaker who is the target of the concentration in the current concentration setting table 24 is changed to the father. And the mother can perform an alternative dialogue instead of the voice recognition dialogue device.
[0065]
During such a dialogue, if the father wants to talk with the voice recognition dialogue device concentrated on himself, the father speaks “Please listen carefully”. As a result, the concentration level control unit 14 changes the concentration level in the concentration level setting table 24 to “high” in accordance with the condition No. 4 in the transition condition table 21 in step S25. Change the target specific speaker to "Father". In this situation, if a mother in the same direction speaks something, the voice input unit 10 captures the voice, but the speaker collated by the speaker collation unit 12 is the mother, and the concentration in the current concentration setting table 24 The speech recognition result of the mother's utterance content is rejected because it does not match the father of the specific speaker who is the target of the degree (S21 is Yes, S22 is No, S23), and conversation can be concentrated with the father. It becomes like this. When the concentration level in the concentration setting table 24 is “high”, the directivity is adjusted to −45 degrees to 45 degrees according to the definition content of the concentration level “high” in the concentration definition table 22. Therefore, the voice input unit 10 also reduces the probability of uttering a person's utterance or noise that is not related to another direction, makes it easier to capture the father's voice, and improves the voice recognition rate.
[0066]
Next, in this situation, if the father makes an utterance that does not match the condition content of the concentration level in the concentration level transition condition table 21 such as “but yesterday” (No in S24), the concentration level setting is performed. The concentrated conversation state is maintained with the level of concentration of the table 24 being “high”.
[0067]
Next, under this circumstance, if the father wants to stop the conversational state in which the father is concentrated, the father speaks "I'm fine". As a result, the concentration control unit 14 changes the concentration level in the concentration setting table 24 to “low” according to the condition No. 2 in the transition condition table 21 in step S25, and the voice input unit 10 in step S27. Is adjusted to -180 degrees to 180 degrees. The level of concentration in the concentration level setting table 24 is “low” indicating a state in which the concentration level capable of capturing utterances from any direction is diverged, so that the speech recognition results other than the specific speaker will not be rejected from the next time. Valid (No in S21).
[0068]
Further, even if a father who is a specific speaker whose identification name is set in the concentration level setting table 24 leaves the place in a concentrated conversation state with a high level of concentration level, FIG. As already described with reference to the flowchart, if the time when there is no speech continues for 30 seconds, the concentration level in the concentration setting table 24 changes to “medium” due to the condition No. 7 in the concentration change condition table 21. However, if the time when there is no further utterance continues for 30 seconds, the concentration level in the concentration level setting table 24 changes to “low” due to the condition No. 6 in the concentration level change condition table 21, so that the mother and other speakers Can also interact with the speech recognition dialogue device.
[0069]
Next, the effect of this embodiment will be described.
[0070]
In the present embodiment, in a situation where a plurality of speakers are in different directions or in the same direction, the concentration level on the speaker is controlled by the concentration level control unit 14 on the speaker. It is possible to switch naturally, such as conversing with only a speaker, and sometimes having an alternative dialogue with multiple speakers.
[0071]
In addition, the probability of picking up utterances and noises of other unrelated people during a dialogue with a specific speaker can be reduced during the dialogue.
[0072]
Other Embodiments of the Invention
FIG. 4 is a block diagram showing a second embodiment of the present invention. Referring to FIG. 4, the second embodiment of the present invention is different from the first embodiment shown in FIG. 1 in that an image input unit 40 is added, instead of the speaker position specifying unit 11. The speaker position specifying unit 41 is provided with a speaker verification unit 42 instead of the speaker verification unit 12. In addition, the same code | symbol as the other FIG. 1 represents the same part.
[0073]
The image input unit 40 has a function of capturing image information in a range of 360 degrees, and is realized by, for example, a plurality of CCD cameras.
[0074]
The speaker position specifying unit 41 has a function of specifying the direction of the speaker who has spoken based on the audio information input from the audio input unit 10 and the image information input from the image input unit 40.
[0075]
The speaker verification unit 42 has a function of specifying a speaker based on the voice information from the voice input unit 10 and the image information from the image input unit 40.
[0076]
Next, the operation of the present embodiment will be described.
[0077]
When the voice information is input from the voice input unit 10, the speaker position specifying unit 41 first specifies the direction of the speaker who speaks based on the voice information. Thereafter, the speaker position specifying unit 41 obtains directions of all speakers around the voice recognition dialogue apparatus based on the image information input by the image input unit 40. Then, among the directions of each speaker obtained based on the image information, the direction closest to the direction of the speaker obtained based on the speech information is set as the direction in which the speaker is speaking, and the direction is controlled by the degree of concentration. To the unit 14.
[0078]
When voice information is input from the voice input unit 10, the speaker verification unit 42 identifies a speaker who has spoken based on the voice information. Further, the speaker verification unit 42 analyzes the image input unit 40 to recognize the speaker whose mouth is moving, and the speaker's face image and a plurality of speaker face images registered in advance. Is used to identify the speaker who spoke. When the speaker specified by the voice information matches the speaker specified by the image information, collation speaker information indicating the speaker is output to the concentration control unit 14, and when they do not match, The collation speaker information indicating the speaker specified by the image information is output to the concentration control unit 14.
[0079]
Since operations other than those described above are the same as those in the first embodiment, description thereof is omitted here.
[0080]
As described above, the present embodiment includes the image input unit 40 such as a camera in addition to the audio input unit 10 such as a microphone array, and the direction of the speaker who speaks based on both the audio information and the image information. Since the speaker who spoke is recognized, the recognition accuracy can be increased.
[0081]
FIG. 5 is a block diagram showing a third embodiment of the present invention. Referring to FIG. 5, in the third embodiment of the present invention, a voice model database 51 is added to the configuration of the first embodiment shown in FIG. The difference is that the unit 52 is provided, and that the concentration control unit 53 is provided instead of the concentration control unit 14. In addition, the same code | symbol as the other FIG. 1 represents the same part.
[0082]
In the voice model database 51, a voice model and a standard voice model of each speaker who uses the voice recognition dialogue apparatus are registered. These are used when performing speech recognition.
[0083]
The concentration level control unit 53 sets the specific speaker identification name set in the concentration level setting table 24 in the speech model database 51 in addition to the functions provided in the concentration level control unit 14.
[0084]
When performing speech recognition, the speech recognition unit 52 uses the speech model of the speaker corresponding to the specific speaker identification name set by the concentration level control unit 53 among the speech models in the speech model database 51. Perform voice recognition. By doing in this way, there exists an effect which can improve the speech recognition rate of the specific speaker used as the object of concentration. When the specific speaker identification name is “unknown”, the speech recognition unit 52 performs speech recognition using a standard speech model.
[0085]
FIG. 6 is a block diagram showing an example of a hardware configuration of the speech recognition dialogue apparatus according to the present invention, which is composed of a computer 61, a recording medium 62, a voice input unit 63, a voice output unit 64, and a database 65. Has been. The voice input unit 63, the voice output unit 64, and the database 65 correspond to the voice input unit 10, the voice output unit 19, and the database 20 shown in FIG. The recording medium 62 is a disk, semiconductor memory, or other recording medium, and stores a program for causing the computer 61 to function as a part of the voice recognition dialogue apparatus. This program is read by the computer 61 and its operation is controlled so that the speaker position specifying unit 11, the speaker verification unit 12, the speech recognition unit 13, and the concentration level control unit 14 shown in FIG. The voice input control unit 15, the other event management unit 16, the dialogue control unit 17, and the voice synthesis unit 18 are realized.
[0086]
【The invention's effect】
The first effect is that when multiple speakers are around the speech recognition dialogue device, particularly when there are multiple speakers in different directions, sometimes the conversation is concentrated only with a specific speaker. However, sometimes it is possible to switch naturally, such as having an alternative dialogue with multiple speakers.
[0087]
The reason is that the level of concentration with respect to the speaker can be determined according to the content of the speaker's utterance, and the directivity and direction of the voice input unit such as a microphone array can be adjusted according to the level of concentration. It is.
[0088]
The second effect is that when there are multiple speakers around the speech recognition dialogue device, especially when there are multiple speakers in the same direction, sometimes the conversation is concentrated only on a specific speaker. In other words, it is possible to switch naturally, such as having a dialogue with multiple speakers.
[0089]
The reason is that the level of the degree of concentration on the speaker is determined according to the content of the speaker's utterance, and the utterances of speakers other than the specific speaker can be invalidated according to the level of concentration.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration example of a first exemplary embodiment of the present invention.
FIG. 2 is a flowchart showing a processing example of a concentration control unit 14 when there is an input from a speaker position specifying unit 11, a speaker collation unit 12, and a voice recognition unit 13.
FIG. 3 is a flowchart showing a processing example of the concentration level control unit 14 when notified from another event management unit 16;
FIG. 4 is a block diagram showing a configuration example of a second embodiment of the present invention.
FIG. 5 is a block diagram showing a configuration example of a third embodiment of the present invention.
FIG. 6 is a block diagram illustrating an example of a hardware configuration of a speech recognition dialogue apparatus.
FIG. 7 is a block diagram for explaining a conventional technique.
[Explanation of symbols]
10 Voice input part
11 Speaker position identification part
12 Speaker verification section
13 Voice recognition unit
14 Concentration control unit
15 Voice input control unit
16 Other Event Management Department
17 Speaker control unit
18 Speech synthesis unit
19 Audio output unit
20 database
21 Transition condition table
22 Definition table
23 Information table
24 Concentration setting table
40 Image input section
41 Speaker position identification part
42 Speaker verification
51 Voice model database
52 Voice recognition unit
53 Concentration control unit
61 computers
62 Recording media
63 Voice input part
64 Audio output unit
65 database
70 Voice input part
71 Voice input control unit
72 Speech feature vector extraction unit
73 Voice recognition unit
74 Recognition result display
75 Image information input section
76 Image Information Analysis Department

Claims (12)

An audio input unit for capturing audio information;
A speaker position specifying unit for specifying the direction of the speaker who has spoken,
A speaker verification unit that identifies the speaker who spoke,
A voice recognition unit that analyzes voice information input from the voice input unit and recognizes voice;
A speaker concentration control unit that controls the speaker concentration;
A voice input control unit that adjusts the input state of the voice input unit according to the level of concentration,
A speech recognition dialogue apparatus comprising: a speaker that stores information necessary for a concentration level control unit for a speaker to control the concentration level, and is referred to and updated, and a concentration level management database. An audio input unit for capturing audio information;
A speaker position specifying unit for specifying the direction of the speaker who has spoken,
A speaker verification unit that identifies the speaker who spoke,
A voice recognition unit that analyzes voice information input from the voice input unit and recognizes voice;
A concentration level setting table in which a specific speaker identification name indicating a specific speaker and a concentration level for the specific speaker indicated by the specific speaker identification name are set;
It is determined whether to enable the speaker's speech based on the content of the concentration level setting table and the speaker specified by the speaker verification unit. The level in the concentration setting table and the specific speaker using the level of concentration determined based on the recognition result of the voice recognition unit for the utterance of the voice and the identification name of the speaker specified by the speaker verification unit Concentration level for updating the identification name and controlling the directivity and direction of the voice input unit based on the content of the updated concentration level setting table and the direction of the speaker specified by the speaker position specifying unit A control unit;
A voice recognition dialogue apparatus comprising: a voice output unit that outputs a response to a speech recognition result determined to be valid by the concentration level control unit. The speech recognition dialogue apparatus according to claim 2,
Another event management unit for detecting that a predetermined event has occurred, and
The concentration control unit is configured to change a concentration level set in the concentration setting table when the occurrence of the predetermined event is detected by the other event management unit. Speech recognition dialogue device. The speech recognition dialogue apparatus according to claim 3,
The predetermined event is that there is no utterance for a predetermined time by a specific speaker whose specific speaker identification name is set in the concentration setting table; and
The concentration control unit detects the occurrence of the predetermined event in the other event management unit, and the concentration level set in the concentration setting table is set in the concentration setting table. If the utterance of a specific speaker indicated by a specific speaker identifier is high enough to enable the utterance by another speaker, the level of concentration set in the concentration setting table is also effective. A speech recognition dialogue apparatus, characterized by having a configuration that is lowered to a level to achieve. The speech recognition dialogue apparatus according to any one of claims 1 to 4,
The voice recognition dialogue apparatus, wherein the voice input unit is composed of a microphone array that can change directivity. The speech recognition dialogue apparatus according to any one of claims 1 to 5,
The speech recognition interactive apparatus characterized in that the speaker position specifying unit specifies a direction of a speaker who speaks based on voice information input by the voice input unit. The speech recognition dialogue apparatus according to any one of claims 1 to 6,
The speech recognition dialogue apparatus, wherein the speaker verification unit has a configuration for identifying a speaker who has spoken based on voice information input by the voice input unit. The speech recognition dialogue apparatus according to any one of claims 1 to 5,
An image input unit for capturing image information; and
The speech recognition characterized in that the speaker position specifying unit specifies a direction of a speaker who speaks based on voice information input by the voice input unit and image information input by the image input unit. Interactive device. The speech recognition dialogue apparatus according to any one of claims 1 to 5,
An image input unit for capturing image information; and
The speech recognition dialogue apparatus characterized in that the speaker verification unit has a configuration for identifying a speaker who speaks based on voice information input by the voice input unit and image information input by the image input unit. The speech recognition dialogue apparatus according to any one of claims 1 to 5,
It has a voice model database in which the voice models of multiple speakers are registered,
The voice recognition unit uses the voice model of a specific speaker whose specific speaker identifier is set in the concentration setting table among the voice models of each speaker registered in the voice model database. A speech recognition dialogue apparatus having a configuration for performing recognition. The speech recognition dialogue apparatus according to claim 1.
The speaker and concentration management database is:
A concentration condition transition condition table storing condition contents for changing the level of concentration on the speaker;
A concentration level definition table that defines the directivity and direction of the microphone array, etc. for each level of concentration level,
An information table for a speaker to be collated when the speaker speaks and a collation speaker information table for storing the position information;
A current concentration level setting table for storing the currently set concentration level and the target speaker information;
A speech recognition dialogue apparatus having a configuration capable of referring to and updating necessary information for controlling the degree of concentration on a speaker. A program for causing a computer equipped with a voice input unit for capturing voice information to function as a voice recognition dialogue apparatus,
The computer,
A speaker position specifying unit for specifying the direction of the speaker who has spoken,
Speaker verification unit that identifies the speaker who spoke,
A voice recognition unit that analyzes voice information input from the voice input unit and recognizes a voice;
The specific speaker identification name indicating the specific speaker and the content of the concentration level setting table in which the level of the concentration level for the specific speaker indicated by the specific speaker identification name is set, and the speaker specified by the speaker verification unit Whether or not to enable the speaker's utterance based on the level of concentration determined based on the recognition result of the voice recognition unit for the speaker's utterance And the speaker identification name specified by the speaker verification unit, the level and the specific speaker identification name in the concentration setting table are updated, and the content of the updated concentration setting table and the talk are updated. A concentration control unit that controls the directivity and direction of the voice input unit based on the direction of the speaker specified by the speaker position specifying unit;
A program for functioning as a voice output unit that outputs a response to a recognition result of an utterance determined to be valid by the concentration level control unit.

Images