JP5701935B2 - Speech recognition system and method for controlling speech recognition system - Google Patents

Description

  The present invention relates to a speech recognition system and a control method for the speech recognition system.

  In a dialogue between people, there is no single response (reply) to one word, and various responses are made according to the situation. Therefore, it has been devised so that various responses can be made to one word even in a dialogue between a person and a robot (Patent Documents 1-4).

  In Patent Document 1, an input sentence from a user is classified, and a response sentence is generated according to a rule for each classification. In the prior art described in Patent Document 1, to classify the input sentence with the word, for example, "Hello", "hello" changes the reply in response to the emotion of that time (Fig. 8 of Patent Document 1). Further, the input sentence can be classified into “greeting”, “question”, “impression”, etc., so that the question is not returned in response to the question from the user (FIG. 21 of Patent Document 1).

  In Patent Document 2, the structure of the input sentence is analyzed, and it is determined whether or not the word extracted from the input sentence is an appropriate word to ask for emotion, frequency, and parallel object, and is determined to be appropriate. If so, a response sentence is generated.

  In Patent Document 3, a plurality of responses are generated according to a response generation rule for an utterance of a conversation partner, and one response is selected from the generated responses based on a predetermined rule.

  In Patent Document 4, when a response sentence includes words that can be used according to the positional relationship, a response sentence is generated by selecting a word corresponding to the current positional relationship.

JP 2009-151314 A JP 2010-224608 A JP 2003-255990 A JP 2001-188551 A

  The prior art of Patent Document 1 classifies input sentences into types such as “greeting”, “question”, and “impression”, but the classification is used to maintain the dependency between the input sentence and the response sentence. It is difficult to achieve a natural response.

  The conventional technology described in Patent Document 2 classifies the question format according to the extracted word and generates a response sentence suitable for the question format. However, since the response depends on the structure of the sentence, a natural response is obtained. It is difficult to realize.

  Since the prior art described in Patent Document 3 selects a response with a probability, it is difficult to realize a natural response reflecting the state of dialogue.

  The prior art described in Patent Document 4 simply responds in consideration of the positional relationship, and it is difficult to realize a natural response.

  As described above, in the technique of selecting a response with a probability from a plurality of responses registered in advance for one word, it is not possible to correctly reflect the situation and respond. Also, in the technology that responds according to the scenario, it is necessary to create scenarios corresponding to various situations in advance, and this work is very laborious.

  By the way, even if the expressions of the responses are different, it is natural to feel the same reaction as long as the meaning and situation are the same because there is a sense of unity.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a voice recognition system and a voice that can realize a system capable of various responses with a sense of unity according to a voice input sentence relatively easily. It is to provide a control method of a recognition system. Another object of the present invention is to provide a voice recognition system and a control method for the voice recognition system that can provide a unified response according to the meaning and type of the voice input sentence.

  A system according to one aspect of the present invention is a speech recognition system that recognizes and responds to speech, and recognizes a speech input sentence input from the speech input section, and a speech input sentence by the speech recognition section. Based on the recognition result and predetermined information, a determination unit for determining whether the voice input sentence is a voice input to the own system, and a plurality of input sentence types prepared in advance. A plurality of reactions prepared in advance for each input sentence type based on the input sentence classification unit to be classified, the input sentence type into which the voice input sentence is classified, and the determination result on the voice input sentence by the determination unit A response level determination unit for determining a response level for a voice input sentence from the levels, a response determination unit for determining a response to the voice input sentence from predefined responses based on the determined response level, Includes a response output unit for outputting a constant and response, the.

  The input sentence classification unit classifies the voice input sentence into any of a plurality of meaning groups prepared in advance according to the meaning of the sentence, and further, a plurality of correspondences associated with one or a plurality of meaning groups Among the input sentence types, the input sentence types can be classified into the input sentence types corresponding to the meaning group to which the voice input sentences are classified.

  An image output unit that outputs an image of a predetermined range around the own system, and the reaction level determination unit has a plurality of response levels based on the input sentence type, the determination result, and the analysis result of the image output by the image output unit. The reaction level for the voice input sentence can be determined from the inside.

  The response determination unit can determine a response to the voice input sentence based on the determined response level and the analysis result of the image output from the image output unit.

The block diagram which shows the structure of a speech recognition system. Explanatory drawing which shows the method of extracting the some parameter for evaluating the reliability of a user's utterance. Explanatory drawing which shows the correlation (a) of a keyword and a verb, and the relationship (b) between keywords. Explanatory drawing which shows the example of the database for classifying an input sentence. Explanatory drawing which shows the example of the database which determines the reaction level of a response according to the classification and reliability of an input sentence. Explanatory drawing which shows the example of the database which determines the content of a response according to the meaning of an input sentence, a reaction level, etc. The flowchart which shows the whole process until it recognizes and responds to an audio | voice. The flowchart which shows the process which classifies an input sentence.

  In the present embodiment, as described in detail below, the types of voice input sentences (for example, greetings, conversations, conversations, etc.) are classified according to the situation, and the general degree of reaction (strength) depending on the situation Define In the present embodiment, the voice input sentences are further grouped according to their meanings, and a response corresponding to the degree of reaction (and the situation using the image analysis result) is defined in advance for each meaning group. Thereby, the speech recognition system of this embodiment can respond variously with an appropriate degree of reaction according to the type and situation of the speech input sentence.

  Furthermore, the present system does not need to prepare a scenario for each individual scene, and can be realized with a relatively simple program, thereby reducing the construction cost of the present system.

  Further, in this embodiment, voice input sentences are grouped according to the meaning of the sentence, and the type of the input sentence is determined for each sentence meaning, so even when a new word appears, the voice input sentences are grouped according to the meaning of the sentence. Just register a new word.

  FIG. 1 is a block diagram showing the overall configuration of the speech recognition system 1 of the present embodiment. The voice recognition system 1 is configured as a computer system. The voice recognition system 1 can be mounted on an object 2 having various shapes such as a robot shape or a cylindrical shape or a rectangular parallelepiped. Furthermore, the voice recognition system 1 can be mounted on various electrical products 3 such as a display device, a vacuum cleaner, a refrigerator, a mobile phone, a portable information terminal, a home energy management system (HEMS), and a washing machine. In this embodiment, a case where the voice recognition system 1 is mounted on a humanoid robot 2 will be described as an example.

  Note that the entire voice recognition system 1 may be provided in the robot 2, or a part of the voice recognition system 1 may be provided outside the robot 2. Furthermore, a configuration in which almost the entire speech recognition system 1 is provided in an external server, and a man-machine interface (speech input unit 21, camera 23, speech output unit 41, display unit 42) for exchanging information with the user is provided in the robot 2. But you can. However, by providing the entire voice recognition system 1 in the robot 2, it is possible to prevent a time lag from occurring in the conversation with the user and realize more natural communication.

  The speech recognition system 1 includes, for example, a speech recognition unit 11, an utterance reliability determination unit 12, an input sentence classification unit 13, a reaction level determination unit 14, a response determination unit 15, and a response output unit 16. Furthermore, the speech recognition system 1 includes a speech input unit 21, an image analysis unit 22, a camera 23, an acoustic model database 24, a grammar database 25, a dictionary database 26, a keyword-verb database 27, a recognition result history database 28, and a speech output unit 41. A display unit 42 and an operation mechanism 43.

  The audio input unit 21 includes, for example, one or a plurality of microphone devices and an A / D (Analog / Digital) conversion circuit. In this embodiment, it is assumed that a sound source direction microphone capable of detecting the direction of a sound source is also included in the voice input unit 21.

  The image analysis unit 22 analyzes image data captured by the camera 23 which is an example of the “image output unit”, and outputs an analysis result. For example, the image analysis unit 22 determines whether the captured image data includes a face, whether the captured face is a front face or a side face, and whether the reflected face matches a user's face registered in the system 1. Or analyze. The image analysis result is used by the utterance reliability determination unit 12, the reaction level determination unit 14, and the response determination unit 15.

  The camera 23 is attached to, for example, the head or chest of the robot 2 and photographs a user or the like existing around the robot 2 and outputs image data. The camera 23 is not necessarily attached to the robot 2. A configuration may be employed in which a camera 23 is provided at a location away from the robot 2 and image information captured by the camera 23 is transmitted to the voice recognition system 1 via wireless communication or the like. Note that the image data captured by the camera 23 and / or the image analysis result of the image analysis unit 22 can be accumulated for a predetermined time.

  The voice recognition unit 11 has a function of analyzing the voice input from the voice input unit 21 using the acoustic model database 24, the grammar database 25, and the dictionary database 26, and converting them into words.

  The acoustic model database 24 is a database that stores text (reading) and a waveform when the text is pronounced in association with each other, and defines what kind of waveform sound is recognized as what word. . The grammar database 25 is a database that stores word arrangement (grammar) and the like. In the dictionary database 26, various words including predetermined keywords are registered together with their readings. The history of the speech recognition result by the speech recognition unit 11 is stored in the recognition result history database 28.

  The utterance reliability determination unit 12 extracts a predetermined parameter serving as information for determining the utterance reliability. The utterance reliability determination unit 12 uses the recognition result of the speech recognition unit 11, the image information captured by the camera 23, the sound source direction information, the keyword-verb database 27, and the recognition result history database 28 to set predetermined parameters. get.

  Furthermore, the speech reliability determination unit 12 determines whether the recognition result by the voice recognition unit 11 is a voice from a user directed to the own system based on a predetermined parameter. Specifically, the utterance reliability determination unit 12 determines whether the input is a voice input from the user to the own system or a voice instruction to the own system. The voice instruction is basically configured as a combination of a predetermined keyword and a predetermined verb. However, the present invention is not limited to this, and it can be determined that only a predetermined keyword is a voice instruction.

  The input sentence classification unit 13 is prepared in advance in the input sentence classification database 29 with the voice input sentence recognized by the voice recognition unit 11 and determined by the utterance reliability determination unit 12 to have an utterance reliability equal to or higher than the lower limit value. The input sentence is classified into one of a plurality of input sentence classifications.

  The reaction level determination unit 14 searches the reaction level database 30 based on the classification result of the voice input sentence and the utterance reliability, so that the reaction level corresponding to the voice input sentence is selected from the reaction levels prepared in advance. Select one.

  The response determination unit 15 searches the response definition database 31 based on the reaction level determined by the reaction level determination unit 14 and the voice input sentence, thereby determining a response to the voice input sentence from the predefined responses. To do.

  The response output unit 16 executes the response determined by the response determination unit 15. The response output unit 16 can respond to the user using, for example, one or more of the audio output unit 41, the display unit 42, and the operation mechanism 43.

  The voice output unit 41 is composed of, for example, a synthesized voice output device and a speaker, and responds to the user with voice. The display unit 42 includes, for example, a lamp and a display provided in the robot 2 and displays a message or turns on the lamp. The operation mechanism 43 is, for example, a mechanism for moving the robot's neck, limbs, and the like. In addition to audio output, display output, and operation, output to a printer or the like may be performed.

  The predetermined parameters extracted by the utterance reliability determination unit 12 will be described with reference to FIG. The utterance reliability determination unit 12 can extract all or part of the first parameter, the second parameter, and the third parameter as described below. The utterance reliability determination unit 12 need not always detect all the first to third parameters. Further, the method for determining the speech reliability described below is an example, and the present invention is not limited to the following method. Hereinafter, a voice input sentence may be abbreviated as an input sentence.

  The first parameter shown in FIG. 2A is obtained by analyzing the latest input sentence input from the voice input unit 21 to the voice recognition unit 11. The first parameter can further include a plurality of (for example, three) subparameters (1A) to (1C).

  The first subparameter (1A) is a rate of a predetermined keyword (and a predetermined verb) in the latest input sentence. The voice recognition unit 11 recognizes the voice (user's voice instruction, surrounding conversation, ambient noise, etc.) input to the voice input unit 21 by applying it to the dictionary database 26. The level of the utterance reliability can be determined from the ratio of the predetermined keyword (and the predetermined verb) included in the recognized word group.

  The predetermined keyword is a keyword indicating a service that can be provided by the speech recognition system 1 (specifically, a service that can be provided by the system (the robot 2) in which the speech recognition system 1 is installed) among general keywords. It is. Furthermore, the predetermined keyword includes a keyword that may be issued when the user uses the service, for example, a keyword corresponding to a reply such as “Yes” or “No”.

  The predetermined keyword is registered in advance in the speech recognition system 1 as a keyword that serves as a voice instruction when using the service. Specifically, keywords relating to services that can be provided to the user by the robot 2 on which the voice recognition system 1 is mounted are registered in the voice recognition system 1 as predetermined keywords.

  For example, if the robot 2 can provide services for dancing, reading news and stories, giving quizzes, predicting the weather, and fortune-telling, a keyword (“ Dance, News, Story, Quiz, Weather, Fortune-telling) are registered as predetermined keywords.

  Reference is first made to FIG. FIG. 3A is an example of a table T10 that manages association between a predetermined keyword and a predetermined verb. This table T10 is an example of the keyword-verb database 27 shown in FIG.

  A predetermined verb related to each keyword is associated with the predetermined keyword in advance. For example, in the case of the keyword “dance”, “do”, “show”, “dodori”, and the like can be considered as predetermined verbs frequently used with the keyword. In the case of the keyword “news”, “Yonde”, “Kisekete”, “Takeshite”, etc. can be considered as predetermined verbs.

  Returning to FIG. The utterance reliability can be determined from the ratio of a predetermined keyword occupying the words constituting the latest input sentence extracted as the first subparameter (1A).

  For example, when both a predetermined keyword and a predetermined verb are included in the latest input sentence, it can be determined that the utterance reliability is high. For example, when only a predetermined keyword is included in the latest input sentence and there is no predetermined verb, it can be determined that the utterance reliability is medium. For example, when a predetermined keyword is not included, or when the proportion of meaningless words (non-keywords) other than the predetermined keyword is larger than a predetermined reference value, it can be determined that the utterance reliability is low.

  The second subparameter (1B) of the first parameter will be described. In the second subparameter (1B), it is determined whether a predetermined keyword is included based on the relationship between the average value of the sound intensity of the entire latest input sentence and the sound intensity of each word.

  “Uu” shown in FIG. 2 (1) is a mechanical sound (servo sound) generated by, for example, an air conditioner, a refrigerator, or a washing machine, and is a meaningless non-keyword. Since the speech recognition unit 11 tries to convert the input speech data into a certain word as much as possible, even a simple mechanical sound is recognized as a certain word such as “Uu”.

  However, since the mechanical sound is not an utterance from the user but merely an environmental noise, the sound level is weak. On the other hand, the words spoken by the user have a relatively strong sound level. In particular, when the user issues a predetermined keyword in anticipation of the response of the robot 2, the sound level is considered to be relatively strong.

  Therefore, in the second subparameter (1B), the average value of the sound intensity of the words in the entire input sentence is compared with the sound intensity of each word, and only the predetermined keyword that is strongly generated is determined as the predetermined keyword. Judge that there is. In the illustrated example, the level of the sound of “dance” is 3000, the level of the sound of “do” is 1000, and each is clearly more than the sound intensity of other words (450 to 600). strong. On the other hand, even a word recognized as a predetermined keyword is ignored if its sound level is weaker than the average sound intensity of the entire input sentence. This is because a keyword having a low sound level is considered to have been erroneously recognized accidentally by a combination of ambient noise and the like.

  The third subparameter (1C) of the first parameter will be described. In the third subparameter (1C), it is determined whether or not the word is a word issued by the user based on the sound length (frame length) of each word constituting the latest input sentence. In other words, the third subparameter (1C) removes a recognition result such as noise based on the length of the sound. The length of one frame is obtained from the sampling frequency of the audio input unit 21.

  When a user utters a predetermined keyword, a predetermined time is required due to the characteristics of the real voice. On the other hand, when ambient noise and the like are accidentally combined and misrecognized as a predetermined keyword, it often has a time shorter than the time that a person can speak. Therefore, in the third subparameter (1C), the word having an unnatural length is removed from the recognition result of the latest input sentence, and the word having the natural length is determined as the word originated by the user.

  FIG. 2 (2) shows the second parameter. The second parameter can be obtained by analyzing a recognition result for a predetermined period (for example, about several seconds to several tens of seconds) from the present time. The recognition result within a predetermined period may be considered equally, or the weight may be determined so as to become a past recognition result. For example, if the predetermined period is 7 seconds, words recognized 7 seconds to 5 seconds before are 0.4 words, words recognized 4 seconds to 2 seconds ago are 0.7, 1 second and last. The recognized word may be determined by multiplying by 1.0.

  The second parameter may include a plurality of (for example, four) subparameters (2A) to (2D) as described below.

  In the first sub-parameter (2A) of the second parameter, when a predetermined keyword is detected after continuous mechanical sounds such as “Uu”, it is determined that the user has spoken. That is, when a predetermined keyword is issued in an environment where only mechanical sounds can be heard, it is determined that the word is from the user.

  In the second sub-parameter (2B) of the second parameter, when irrelevant words are continuously recognized, it is determined that the conversation is being performed around the robot 2 instead of a voice instruction of the user. Examples of surrounding conversations include conversations between humans, conversations between humans and other machines (other machines that can be controlled by voice instructions), and audio from televisions or radios.

  FIG. 3B schematically shows the relationship between keywords. Around the predetermined keyword, there are one or more related keywords that are semantically related to the predetermined keyword. For example, in the case of a predetermined keyword “quiz”, related keywords such as “question” and “answer” exist in the vicinity thereof. Furthermore, there are other keywords that are less relevant to the predetermined keyword “quiz”. Other less relevant keywords include other predetermined keywords, keywords related to other predetermined keywords, and the like. In this way, by analyzing the relevance between keywords (words) in advance, it is possible to determine whether or not irrelevant words are continuously input.

  Returning to FIG. In the third sub-parameter (2C) of the second parameter, it is determined that the utterance reliability is high when the appearance frequency of the same predetermined keyword is large. For example, when the same predetermined keyword appears frequently such as “dance”, “dance”, and “dance”, there is a high possibility that the user is speaking. In particular, in the case of a user whose speech is not clear, such as an infant or an elderly person, there is a possibility that only a predetermined keyword is called continuously without a verb.

  In the fourth sub-parameter (2D) of the second parameter, when a predetermined keyword is detected after a predetermined silent period has elapsed, it is determined that the voice instruction is given from the user to the robot 2. That is, the fourth subparameter (2D) is a modified example of the first subparameter (2A), and when a predetermined keyword is detected in a relatively quiet environment, it is determined that the instruction is a voice instruction from the user. To do.

  The third parameter will be described. The third parameter shown in FIG. 2 (3) is extracted from the image information captured by the camera 23 and the sound source direction information specified by the sound source direction microphone included in the audio input unit 21. The third parameter can include a plurality (eg, three) subparameters.

  The first subparameter (3A) is the presence or absence of a face. When the camera 23 captures a human face, there is a high possibility that the recognition result at that time is a voice instruction from the user. The second subparameter (3B) is the face orientation. When the front face of the user is photographed by the camera 23, it is highly likely that the user is talking to the robot 2. The third subparameter (3C) is a sound source direction. When the voice is emitted from the front direction of the robot 2, there is a high possibility that it is a voice instruction from the user.

The utterance reliability determination unit 12 may calculate the utterance reliability in consideration of all the parameters described above, or may calculate the utterance reliability based only on one or more parameters among the parameters. May be. The utterance reliability may be calculated by weighting the determination result with each parameter, or simply calculated as the ratio of the total determination result to the total determination. For example, the utterance reliability may be calculated as a ratio between the number pn of parameters determined to be speech input to the own system and the total number tp of parameters used (utterance reliability = pn / tp).
The utterance reliability is compared with a plurality of predetermined determination values. The system 1 treats an input sentence having only an utterance reliability less than a lower limit value as a first determination value set in advance as an input sentence not to be processed, not an input to the own system. The system 1 determines that the utterance reliability of the input sentence is “normal” when the utterance reliability of the input sentence is equal to or higher than a normal value as a second determination value set in advance. When the utterance reliability of the input sentence is less than the second determination value, the system 1 determines that the utterance reliability of the input sentence is “low”. As described above, in the present embodiment, the utterance reliability of the input sentence is distinguished and treated as any one of “non-target”, “low”, and “normal”, but not limited to this, for example, “non-target”, A configuration may be adopted in which the ranks are handled in a more detailed manner such as “low”, “medium”, and “high”.

  FIG. 4 shows an example of the input sentence classification database 29 used by the input sentence classification unit 13. At the left end in FIG. 4, an input sentence that is a voice recognition result is displayed in katakana. On the right side of FIG. 4, input sentence types indicating the types of input sentences are displayed as, for example, “Talk 1”, “Talk 2”, “Command 1”, and the like. Between the input sentence and the input sentence type, a meaning label for grouping according to the meaning of the input sentence is displayed. The meaning label can also be called a semantic group classified according to the meaning of the input sentence. Since the input sentence type is a group for determining the response level, it can be called a response level determination group or a response control group.

For example, the input sentences “Namaeha”, “Onamaeha”, and “Nantamae” all ask the name of the callee, so the meaning label “What is the name?” Is set. Since each of the input sentences “anatadare”, “daredeska”, and “dare” asks who the callee is, the meaning label “who?” Is set. The meaning labels “What is your name?” And “Who is?” Are associated with “Speak 1“ What is your name? ””, Which is a superordinate concept group.
The input sentence type is a type name (for example, talking, command, greeting, etc.) for classifying the meaning label according to the response level and a representative meaning of one or more meaning labels belonging to the type. It is defined in association with a label. In the above example, both the meaning labels “What?” And “Who?” Are defined as belonging to the representative meaning label “Name?” In the type “Talk 1”.

  The input sentences “Hanjishite”, “Mushishinide”, and “Hanashikiite” are associated with the sentence label “Reply”. The input sentences “Hannoushite” and “Hannoha” are associated with the sentence label “React”. The sentence labels “Respond” and “Respond” are both associated with the input sentence type “Talk 1“ Respond ””.

  The input sentences “Attamai”, “Rikou”, and “Kashikoi” are associated with the sentence label “Head”, and the sentence label is associated with the input sentence type “Talk 2“ Head ””. Yes.

  The input sentence belonging to “Talk 1” and the input sentence belonging to “Talk 2” are both words spoken to the user's system by the user, but are defined as different types because the response levels are different.

  The input sentences “Kyonanichi” and “Hinichioshie” are associated with the sentence label “What day today?”, And the sentence label is associated with the input sentence type “Command 1“ What day today? ””. ing. The type “command” is a type indicating a voice instruction for using a service (for example, dance, news reading, date answer, etc.) installed in the robot 2.

  The input sentences “Dance Odotte”, “Dance Missete” and “Dance Yttete” are associated with the sentence label “Dance Dance”, and the sentence label corresponds to the input sentence type “Command 2“ Dance Dance ””. It is attached. The input sentence belonging to “command 1” and the input sentence belonging to “command 2” are both voice instructions for using the service of the robot 2 but have different response levels (to change the response level). Each is defined as a separate type.

  The input sentences “Hajimemashite” and “Ohatsudes” shown at the bottom of FIG. 4 are associated with the meaning label “Nice to meet you”, and the meaning label is associated with the input sentence type “greeting 1“ Nice to meet you ””. It has been.

  FIG. 4 shows a part of an input sentence that can be recognized by the system 1, and the input sentence that can be recognized is in principle associated with any sentence label. In this embodiment, the input sentences that are the speech recognition results are grouped for each meaning label, and further classified into one of the input sentence types via the meaning label. An input sentence that is not registered in the input sentence classification database 29 may be ignored without responding, or may be classified into an input sentence type set as an initial value.

  When a new input sentence (which may be only a word) is registered in the input sentence classification database 29, it is only necessary to associate it with one of a plurality of sentence meaning labels. The new input sentence is classified into a predetermined input sentence type via the set sentence label. When an appropriate sentence label is not found in the new input sentence, a new sentence label is generated and associated therewith, and the new sentence label is associated with one of the input sentence types.

  When it is desired to change the response level for a registered input sentence, the meaning label to which the input sentence belongs may be associated with the input sentence type of the desired reaction level. For example, when it is desired to change the response level of the input sentence “Hanjishite”, the sentence label “Respond” set for the input sentence can be changed from the input sentence type “Talk 1” to “Talk 2”. Alternatively, a new input sentence type may be defined, and the sentence label to be changed may be associated with the newly defined input sentence type.

  In the input sentence classification database 29, a plurality of input sentences are grouped according to their meanings, and are associated with the input sentence types. Therefore, even if the input sentences have different expressions, they are the same as long as the sentence meanings are the same or similar. It can respond at the reaction level. Thereby, since it can respond with the same reaction level with respect to the input sentence of the similar meaning, the response with a sense of unity can be implement | achieved. Note that the similar sentence meanings include, for example, a plurality of different meaning labels associated with the same input sentence type (such as “What is the name?” And “Who?”, “Respond” and “React”). ).

  Furthermore, since it is possible to classify into the appropriate input sentence type simply by setting the meaning label for the new input sentence, it is not necessary to directly define the reaction level for each input sentence, and it is flexible to new words and expressions. It can respond promptly and is easy to use in system construction and system operation.

  FIG. 5 shows an example of the reaction level database 30. The reaction level database 30 is used by the reaction level determination unit 14, and one or a plurality of reaction levels are set for each input sentence type.

In this embodiment, three levels of “no reaction”, “safe reaction”, and “original reaction” are set as reaction levels. “No response” means no response to the input sentence. “Safe response” means responding to an input sentence, but making an ambiguous or unclear response different from the original response to the input sentence. “Original response” means an original response to an input sentence, a response that is realized as a normal dialogue, or a normal response that a user who is a speaker would expect. .
Each input sentence type has a condition for selecting a part or all of each reaction level. For example, in the case of the input sentence type “talk 1”, the response level “no response” is selected when the utterance reliability is low, and the response level “original response” is selected when the utterance reliability is normal. It is set, and “safe reaction” is not set.

  For the input sentence type “talk 2”, when the utterance reliability is low, the response level “no response” is selected, and when the utterance reliability is normal and the face cannot be detected, the reaction level “safe response” is selected, When the utterance reliability is normal and a face can be detected, the response level “original response” is set to be selected.

  In the case of the input sentence type “command 1”, the response level “original response” is always selected, and the response levels “no response” and “safe response” are not set. When the system 1 recognizes the voice instruction classified as “command 1”, the system 1 executes a response corresponding to the command regardless of the situation.

In the case of the input sentence type “command 2”, when the utterance reliability is low and the face cannot be detected, the reaction level “safe reaction” is selected, and when the utterance reliability is normal and the face can be detected, the reaction level “ The “original reaction” is set to be selected, and the reaction level “no reaction” is not set.
Examples of “safe responses” to voice instructions include, for example, listening back to the contents of the instructions (for example, responding to “doing dance” as “dance?”) Or requesting postponement of execution ("Is it a dance? Please let me get ready").

  In the case of the input sentence type “greeting 1”, the response level “no response” is selected when the utterance reliability is low, and the response level “original response” is selected when the utterance reliability is normal. The reaction level “safe reaction” is not set.

  In the case of the input sentence type “greeting 2”, when the utterance reliability is low and the face cannot be detected, the reaction level “safe reaction” is selected, and when the utterance reliability is normal and the face can be detected, the reaction level “ “Original response” is selected, and “No response” is not set.

  An example of a “safe response” to a greeting is to return a common greeting such as “Hi” or “Yes” that can be received in various ways.

  As described above, the reaction level relating to the response to the input sentence can be determined using the reaction level database 30. Even if the input sentence is completely different, if the response level is the same, it will respond with the same level of strength. Conversely, even if the input sentence is the same, if the situation (whether or not a face has been detected) is different, the reaction level is different, and as a result, the strength of the response also changes.

  FIG. 6 shows an example of the response definition database 31 used by the response determination unit 15. The response definition database 31 defines the response content corresponding to the reaction level or the response content corresponding to the reaction level and the situation for each sentence label.

  When the utterance reliability is normal, the input sentence belonging to the meaning label “What is your name?” Responds with “original response”. The system 1 authenticates the speaker, and responds with "My name is OO" if the speaker is not registered. If the speaker is already registered, it responds like “My name is 〇〇. I thought I knew it”. Whether or not the speaker has been registered in the system 1 can be determined from a user voice registered in advance or a similarity to a face image.

  Since the input sentence belonging to the sentence label “Respond” is classified into the input sentence type “Talk 1”, when the utterance reliability is normal, the “original response” is “ Replying to @Now

  Since the input sentence belonging to the sentence label “smart” is classified into the input sentence type “talk 2”, when the utterance reliability is normal and the face cannot be detected, the “safe response” to the input sentence is given. , “Are you praised?” On the other hand, when the utterance reliability is normal and a face can be detected, a response such as “I am still studying” is given as the “original response”.

  Since the input sentence belonging to the sentence label “How many days today?” Is classified into the input sentence type “command 1”, the input sentence is always “original response” regardless of the situation as “Otsuki”. “It's a day.”

  Input sentences belonging to the meaning label “dance dance dance” are classified into the input sentence type “command 2”, so when the utterance reliability is low and the face cannot be detected, “safe response” is indicated as “dancing? "?" On the other hand, when the utterance reliability is normal and a face can be detected, a dance program stored in advance is executed as an “original reaction” to dance.

  Since the input sentence belonging to the sentence label “Nice to meet you” is classified into the input sentence type “greeting 1”, a response is returned with “original response” when the utterance reliability is normal. The system 1 responds considering not only the reaction level but also the detection status of the voice input sentence. For example, the system 1 determines whether the image analysis unit 22 has detected a face in the image data, performs personal authentication of the speaker, and further determines whether the detected face is a profile. Personal authentication may be performed based on the similarity between the registered user voice and the input sentence, or if this is possible, the degree of coincidence between the registered user's face data and the face data detected by the image analysis unit 22 You may authenticate yourself.

  The first situation is a case where the face cannot be detected and the speaker is an unregistered speaker or an unregistered speaker who can detect only the side face. In the first situation, it responds as “original response,” “Nice to meet you, my name is 00”.

  The second situation is when the front face can be detected and the speaker is not registered. In the second situation, it responds like "Nice to meet you, my name is OO. Please shake your hand."

The third situation is when the front face can be detected and the speaker has already been registered. In the third situation, respond as follows: “Nice to meet you, my name is 〇〇.
As described above, the content of the response can be defined for each sentence label based on the reaction level and the situation. As a result, a uniform response can be returned at a consistent response level, and a natural dialogue can be realized.

  FIG. 7 is a flowchart showing the overall processing of the voice recognition system 1. When the system 1 detects some kind of voice via the voice input unit 21 (S1), it analyzes the inputted voice, extracts the predetermined parameter described in FIG. 2 (S2), and determines the utterance trust based on the predetermined parameter. The degree is determined (S3).

  The system 1 determines whether the utterance reliability calculated for the input sentence is greater than or equal to a predetermined lower limit value (S4), and if it is determined that the utterance reliability is equal to or lower than the lower limit value (S4: NO), the process ends. If the system 1 determines that the input sentence has an utterance reliability equal to or higher than the lower limit value (S4: YES), the input sentence is classified by the meaning label and the input sentence type using the input sentence classification database 29 ( S5).

  The system 1 uses the reaction level database 30 to determine the reaction level for the input sentence type that is the classification destination of the input sentence (S6). The system 1 uses the response definition database 31 to determine the response content based on the sentence label set in the input sentence and the determined reaction level (S7). The system 1 outputs the determined response content using one or more of the audio output unit 41, the display unit 42, and the operation mechanism 43 (S8).

  FIG. 8 is a flowchart showing an example of the input sentence classification process S5 in FIG. The input sentence classification unit 13 of the system 1 searches the input sentence classification database 29 to determine whether there is a sentence label corresponding to the input sentence (S50). When the input sentence classification unit 13 determines that there is a corresponding sentence label (S50: YES), the input sentence classification unit 13 sets the sentence label as an input sentence (S51). Furthermore, the input sentence classification unit 13 determines the input sentence type to which the selected sentence label belongs as the input sentence classification destination (S52).

  According to this embodiment configured as described above, the input sentence is classified into input sentence types according to the meaning of the sentence, and the reaction level is defined for each input sentence type in consideration of the utterance reliability and the situation. The response content is determined for each sentence label according to the situation.

  As described above, in this embodiment, the reaction level determination process (input sentence classification unit 13 and reaction level determination unit 14, steps S5 and S6) is divided from the response content determination process (response determination unit 15, step S7). Since the response level is set for each type of input sentence, a consistent and consistent response can be returned according to the sentence meaning of the input sentence, and a more natural dialogue can be realized.

  Furthermore, since the response content corresponding to the reaction level and the situation is defined for each sentence of the input sentence, a uniform and flexible response can be returned.

  In this embodiment, the input sentence is grouped by setting a meaning label according to the meaning of the sentence, so that new words and expressions can be flexibly and promptly dealt with.

  In this embodiment, the face image analyzed from the image data is used not only as one of the parameters for calculating the utterance reliability but also as the situation determination information for determining the conversation situation. Therefore, as is the case with many actual humans, it is possible to respond by judging the situation based on the presence or absence of a face, etc., and realizing a more natural dialogue.

  The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and changes within the scope of the present invention.

  1: voice recognition system, 2: robot, 3: home appliance, 11: voice recognition unit, 12: utterance reliability determination unit, 13: input sentence classification unit, 14: reaction level determination unit, 15: response determination unit, 16 : Response output unit, 21: Audio input unit, 22: Image analysis unit, 23: Camera, 28: Recognition result history database, 41: Audio output unit, 42: Display unit, 43: Operating mechanism

Claims (9)

A speech recognition system that recognizes and responds to speech,
A speech recognition unit that recognizes a speech input sentence input from the speech input unit;
A determination unit that determines whether the voice input sentence is a voice input to the system based on a recognition result of the voice input sentence by the voice recognition unit and predetermined information;
An input sentence classification unit for classifying the voice input sentence into any of a plurality of input sentence types prepared in advance;
Plural items defined according to the degree of reaction prepared in advance for each input sentence type based on the input sentence type into which the voice input sentence is classified and the determination result of the voice input sentence by the determination unit A reaction level determination unit for determining a reaction level for the voice input sentence from among the reaction levels of
A response determining unit that determines a response to the voice input sentence from among predetermined responses based on the determined response level;
A response output unit for outputting the determined response;
A speech recognition system comprising: The input sentence classification unit classifies the voice input sentence into any of a plurality of meaning groups prepared in advance according to the meaning of the sentence, and further associates the input sentence with one or a plurality of meaning groups. Among a plurality of input sentence types, classify into input sentence types corresponding to the meaning group to which the voice input sentence is classified,
The speech recognition system according to claim 1. An image output unit that outputs an image of a predetermined range around the own system,
The response level determination unit determines a response level for the voice input sentence from the plurality of reaction levels based on the input sentence type, the determination result, and the analysis result of the image output from the image output unit. ,
The speech recognition system according to claim 1. The response determination unit determines the response to the voice input sentence based on the determined response level and the analysis result of the image output by the image output unit;
The voice recognition system according to claim 3. The predetermined information includes sound source direction information indicating an analysis result of the image by the image output unit and / or an input direction of the voice input sentence,
The determination unit includes a first parameter obtained from an analysis result regarding the latest speech input sentence recognized by the speech recognition unit, a second parameter obtained from a recognition result history recognized by the speech recognition unit, and the image Determining based on the analysis result and / or the third parameter obtained from the sound source direction information;
The voice recognition system according to claim 3. The first parameter includes a keyword rate indicating a ratio that a predetermined keyword is included in the latest input sentence, a sound intensity of each word constituting the latest input sentence, and each of the words constituting the latest input sentence. Includes at least one of the lengths of the sounds of the words,
The speech recognition system according to claim 5. The second parameter includes a possibility of indicating whether or not a conversation unrelated to the voice instruction from the user is being performed in the surroundings,
The third parameter includes at least one of whether or not a user's face has been detected and the detected face orientation.
The speech recognition system according to claim 5 or 6. A method for controlling a speech recognition system that recognizes and responds to speech,
A voice recognition step for recognizing a voice input sentence input from the voice input unit;
A determination step of determining whether or not the voice input sentence is a voice input to the own system based on a recognition result of the voice input sentence by the voice recognition step and predetermined information;
An input sentence classification step for classifying the voice input sentence into any of a plurality of input sentence types prepared in advance;
Plural items defined according to the degree of reaction prepared in advance for each input sentence type based on the input sentence type into which the voice input sentence is classified and the determination result of the voice input sentence in the determining step. A reaction level determination step for determining a response level for the voice input sentence from among the response levels of
A response determining step for determining a response to the voice input sentence from among predetermined responses based on the determined response level;
A response output step of outputting the determined response;
For controlling a speech recognition system. A computer program for causing a computer to function as a speech recognition system that recognizes and responds to speech,
A voice recognition unit for recognizing a voice input sentence input from a voice input unit connected to the computer;
A determination unit that determines whether the voice input sentence is a voice input to the system based on a recognition result of the voice input sentence by the voice recognition unit and predetermined information;
An input sentence classification unit for classifying the voice input sentence into any of a plurality of input sentence types prepared in advance;
Plural items defined according to the degree of reaction prepared in advance for each input sentence type based on the input sentence type into which the voice input sentence is classified and the determination result of the voice input sentence by the determination unit A reaction level determination unit for determining a reaction level for the voice input sentence from among the reaction levels of
A response determining unit that determines a response to the voice input sentence from among predetermined responses based on the determined response level;
A response output unit for outputting the determined response;
A computer program for realizing the above on the computer.

Images