JP5658641B2 - Terminal device, voice recognition program, voice recognition method, and voice recognition system - Google Patents

Description

  The present invention relates to a terminal device that processes a speech recognition result, a speech recognition program, a speech recognition method, and a speech recognition system.

  There is a technique for recognizing the content of voice input from a microphone and executing the recognized result as a command. For example, Patent Documents 1 and 2 listed below disclose techniques for performing intra-terminal speech recognition and converting the results of intra-terminal speech recognition into commands.

JP 2007-318319 A Japanese Patent Application Laid-Open No. 7-219587

  However, the inventions described in Patent Document 1 and Patent Document 2 have the following problems. That is, in the inventions described in Patent Document 1 and Patent Document 2, if the result of speech recognition is registered in the command dictionary, the command is executed, and if not registered, an error is notified. In terminal speech recognition, the vocabulary that can be recognized is limited, so if a word outside the vocabulary of terminal speech recognition is included in the command, even if the command is spoken correctly, it will not be recognized correctly by speech recognition in the terminal. The command may not be executed.

  Accordingly, an object of the present invention is to provide a terminal device, a speech recognition program, a speech recognition method, and a speech recognition system that execute a command inputted by speech at high speed and in order to solve such problems. To do.

  In order to solve the above problems, a terminal device of the present invention has a voice input unit that receives an input of a voice signal, a voice recognition unit that performs voice recognition on a voice signal received by the voice input unit, and a plurality of commands registered. A command dictionary, a voice transmitting means for sending a voice signal to a voice recognition server, a server voice recognition result receiving means for receiving a server voice recognition result as a voice recognition result for the voice signal by the voice recognition server, and a voice recognition means. The in-terminal speech recognition result, which is the recognized speech recognition result, is checked against the command dictionary to determine which of the in-terminal speech recognition result and the server speech recognition result to use, and the determined speech recognition A command verification unit that identifies a command indicated by the voice signal based on the result, and a command verification unit Characterized in that it and a command executing means for executing the identified command.

  The voice recognition system of the present invention is a voice recognition system including a terminal device and a voice recognition server, and the terminal device is received by a voice input unit that receives an input of a voice signal and a voice input unit. Voice recognition means for performing voice recognition on a voice signal, a command dictionary in which a plurality of commands are registered, voice transmission means for sending a voice signal to a voice recognition server, and a server that is a voice recognition result for the voice signal by the voice recognition server The server speech recognition result receiving means for receiving the speech recognition result, the in-terminal speech recognition result that is the speech recognition result recognized by the speech recognition means is checked against the command dictionary, and the in-terminal speech recognition result and the server speech recognition result Decide which voice recognition result to use and indicate it by the voice signal based on the determined voice recognition result. And a command execution unit that executes the command specified by the command verification unit, and the voice recognition server includes a voice reception unit that receives a voice signal transmitted from the voice transmission unit, A server dictionary that includes more vocabulary than the terminal device, a server speech recognition unit that recognizes speech signals received by the speech receiving unit based on the server dictionary, and a server that transmits a server speech recognition result to the terminal device Voice recognition result transmission means.

  According to the present invention, the in-terminal speech recognition result is collated with the command dictionary, and based on the collation result, by determining which of the in-terminal speech recognition result and the server speech recognition result to use, for example, If the in-terminal speech recognition result is accepted as a command, the command is executed using the in-terminal speech recognition result. If the in-terminal speech recognition result cannot be accepted as a command, the server speech recognition result is used. can do. In other words, when a command that can be recognized by the vocabulary in the terminal is input by voice, a command that cannot be recognized by the vocabulary in the terminal can be input by using the result of speech recognition in the terminal. In this case, the command inputted by voice can be surely recognized and executed by using the server voice recognition result.

  Further, the command collating means collates the in-terminal speech recognition result with the command dictionary, calculates a certainty factor indicating the possibility of corresponding to the command, and when the certainty factor is equal to or greater than a predetermined threshold, the in-terminal speech recognition Preferably, the use of the result is determined, and a command having a certainty level equal to or greater than a threshold is specified as the command indicated by the voice signal. According to this, when the speech recognition is correctly performed even if the reliability of the speech recognition result in the terminal is low, the speech recognition result in the terminal can be used. As a result, a command that can be recognized by the vocabulary in the terminal can be executed at high speed even if the reliability of the speech recognition result in the terminal is low.

  The command dictionary stores a keyword list in which a plurality of keywords and scores associated with the plurality of keywords are registered for each of the plurality of commands. For each of the words included in the keyword, it is determined whether it corresponds to any of a plurality of keywords registered in the keyword list, and the certainty factor is calculated based on the command and score associated with the corresponding keyword. Is preferred. According to this, it can be determined whether or not the command can be recognized by the vocabulary in the terminal, and the command indicated by the voice signal can be more reliably recognized.

  In addition, the command matching unit determines whether each of the words included in the in-terminal speech recognition result corresponds to any of a plurality of keywords registered in the keyword list, and is associated with the corresponding keyword. It is preferable to calculate the certainty factor based on the command and the score and the reliability of speech recognition of the word. According to this, it can be determined whether or not the command can be recognized by the vocabulary in the terminal, and the command indicated by the voice signal can be more reliably recognized.

  In addition, when the command with the certainty level equal to or greater than the threshold is a command for instructing the execution of the in-terminal function, the command matching unit determines use of the in-terminal speech recognition result, and the command with the certainty level equal to or greater than the threshold is When the command is a command other than a command for instructing execution of the in-terminal function, it is preferable to determine use of the server speech recognition result. According to this, it is possible to execute a command for instructing execution of an in-terminal function at high speed using the in-terminal speech recognition result, and reliably execute other commands using the server speech recognition result. Can do.

  Further, it is preferable that the voice transmission means transmits the voice signal to the voice recognition server before the voice recognition means obtains the in-terminal voice recognition result. According to this, the server speech recognition result can be received earlier. For this reason, when it is determined to use the server speech recognition result, the command can be executed at high speed.

  The server speech recognition result receiving unit preferably discards the server speech recognition result when the command verification unit receives the server speech recognition result after the command is specified based on the in-terminal speech recognition result. According to this, a command can be specified based on the in-terminal speech recognition result without waiting for the reception of the server speech recognition result. For this reason, it becomes possible to execute a command recognizable by the vocabulary in the terminal at high speed.

  By the way, the present invention can be described as an invention of a terminal device as described above, and can also be described as an invention of a speech recognition program and a speech recognition method as follows. This is substantially the same invention only in different categories, and has the same operations and effects.

  That is, the speech recognition program of the present invention includes a speech input module that accepts an input of a speech signal, a speech recognition module that performs speech recognition on the speech signal received by the speech input module, and a speech that transmits the speech signal to a speech recognition server. A transmission module, a server speech recognition result receiving module that receives a server speech recognition result that is a speech recognition result for a speech signal by a speech recognition server, and a plurality of in-terminal speech recognition results that are speech recognition results recognized by the speech recognition module Are compared with the registered command dictionary to determine which of the in-terminal speech recognition results and server speech recognition results to use, and is indicated by the speech signal based on the determined speech recognition results. A command matching module that identifies the command Characterized in that it comprises a command execution module for executing the command specified by command verification module.

  Further, the command verification module compares the in-terminal speech recognition result with the command dictionary, calculates a certainty factor indicating the possibility of corresponding to the command, and when the certainty factor is equal to or greater than a predetermined threshold, the in-terminal speech recognition Preferably, the use of the result is determined, and a command having a certainty level equal to or greater than a threshold is specified as the command indicated by the voice signal.

  The command dictionary stores a keyword list in which a plurality of keywords and a score associated with each of the plurality of keywords are registered for each of a plurality of commands. For each word included in the recognition result, determine whether it corresponds to one of a plurality of keywords registered in the keyword list, and calculate the certainty factor based on the command and score associated with the corresponding keyword It is preferable to do.

  In addition, the command matching module determines whether each of the words included in the in-terminal speech recognition result corresponds to any of a plurality of keywords registered in the keyword list, and is associated with the corresponding keyword. It is preferable to calculate the certainty factor based on the command and the score and the reliability of speech recognition of the word.

  In addition, when the command with the certainty level equal to or higher than the threshold is a command for instructing the execution of the function within the terminal, the command verification module determines the use of the in-terminal speech recognition result, and the command with the certainty level equal to or higher than the threshold is When the command is a command other than a command for instructing execution of the in-terminal function, it is preferable to determine use of the server speech recognition result.

  Moreover, it is preferable that the voice transmission module transmits the voice signal to the voice recognition server before the voice recognition result is obtained by the voice recognition module.

  The server speech recognition result receiving module preferably discards the server speech recognition result when the command verification module receives the server speech recognition result after the command is specified based on the in-terminal speech recognition result.

  In addition, the speech recognition method of the present invention includes a speech input step for receiving an input of a speech signal, a speech recognition step for performing speech recognition on the speech signal received in the speech input step, and a speech for transmitting the speech signal to a speech recognition server. A transmission step, a server speech recognition result receiving step for receiving a server speech recognition result that is a speech recognition result for a speech signal by the speech recognition server, and a plurality of in-terminal speech recognition results that are speech recognition results recognized in the speech recognition step. Are compared with the registered command dictionary to determine which of the in-terminal speech recognition results and server speech recognition results to use, and is indicated by the speech signal based on the determined speech recognition results. Command verification step to identify the command to be Characterized in that it and a command execution step of executing a command specified Te.

  ADVANTAGE OF THE INVENTION According to this invention, the command input with the audio | voice can be performed reliably at high speed.

It is a figure which shows the function structure of the speech recognition system which concerns on embodiment of this invention. It is a figure which shows the hardware constitutions of the terminal device of FIG. It is a figure which shows an example of the keyword list which the command dictionary of FIG. 1 memorize | stores. It is a figure for demonstrating the calculation method of the certainty factor in the speech recognition system of FIG. It is a flowchart for demonstrating an example of the command discrimination | determination execution process performed with the terminal device of FIG. 7 is a flowchart for explaining another example of command determination execution processing executed by the terminal device of FIG. 1. 7 is a flowchart for explaining another example of command determination execution processing executed by the terminal device of FIG. 1. It is a figure for demonstrating the command discrimination | determination execution process in the speech recognition system of FIG. It is a figure for demonstrating the command discrimination | determination execution process in the speech recognition system of FIG. 3 is a flowchart for explaining an example of a function / application discrimination call process in the voice recognition system of FIG. 1. It is a flowchart which shows the continuation of FIG. It is a figure for demonstrating the function and application discrimination | determination call process in the speech recognition system of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numerals, and redundant description will be omitted.

  FIG. 1 is a diagram illustrating a functional configuration of the speech recognition system according to the present embodiment. As shown in FIG. 1, a voice recognition system 10 is a system that recognizes a voice signal input to a terminal device 1 by in-terminal voice recognition or network type voice recognition, and includes the terminal device 1 and a voice recognition server. 2 is comprised. The terminal device 1 is a device that executes activation of functions provided in the terminal device 1 and acquisition of information from the outside in response to a command indicated by an audio signal input from an audio input device such as a microphone. The speech recognition server 2 is a device that recognizes the speech signal transmitted from the terminal device 1 to acquire a server speech recognition result and transmits the server speech recognition result to the terminal device 1. The terminal device 1 and the voice recognition server 2 are connected for communication via a network NW.

  Here, intra-terminal speech recognition means speech recognition performed in the terminal device 1 and can respond quickly because communication via the network NW is unnecessary, but it is accurate because there are few vocabularies to be recognized. Voice recognition may not be possible. The network-type speech recognition means speech recognition performed by the speech recognition server 2 and has more vocabulary to be recognized than in-terminal speech recognition and has high speech recognition accuracy, but performs communication via the network NW. The response may be delayed due to a communication delay or the like.

  Functionally, the terminal device 1 has a voice input unit 11 (voice input unit), a voice recognition unit 12 (voice recognition unit), a user dictionary 13, a voice transmission unit 14 (voice transmission unit), and server voice. A recognition result receiving unit 15 (server speech recognition result receiving unit), a command verification unit 16 (command verification unit), a command dictionary 17 and a command execution unit 18 (command execution unit) are provided. The terminal device 1 is a device including a voice input device such as a mobile phone, a smart phone, a PDA (Personal Digital Assistants), a car navigation system, and a notebook PC, and is configured by hardware shown in FIG.

  FIG. 2 is a diagram illustrating a hardware configuration of the terminal device 1. As shown in FIG. 2, the terminal device 1 physically includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102 as a main storage device, a ROM (Read Only Memory) 103, a hard disk, and the like. Auxiliary storage device 104, communication module 105 that is a data transmission / reception device such as a network card, voice input device 106 that is a voice input device such as a microphone, input device 107 that is an input device such as a keyboard or mouse, and an output device such as a liquid crystal display This is configured by hardware such as the output device 108. Each function of the terminal device 1 described with reference to FIG. 1 is obtained by reading predetermined computer software on hardware such as the CPU 101 and the RAM 102 shown in FIG. 2, thereby controlling the voice input device 106 and the input device under the control of the CPU 101. 107, the output device 108 and the like are operated, and data is read and written in the RAM 102 and the auxiliary storage device 104.

  Subsequently, the function of the terminal device 1 will be described with reference to FIG. The voice input unit 11 functions as a voice input unit that receives an input of a voice signal. Specifically, the voice input unit 11 receives a voice signal based on the content of the user's utterance input via the voice input device 106 such as a microphone. Then, the voice input unit 11 transmits the received voice signal to the voice recognition unit 12 and the voice transmission unit 14.

  The voice recognition unit 12 functions as a voice recognition unit that performs voice recognition on the voice signal received by the voice input unit 11. The speech recognition unit 12 refers to an acoustic model and a language model stored in advance and a user dictionary 13 described later, and acquires an in-terminal speech recognition result that is a speech recognition result of the terminal device 1. This in-terminal speech recognition result includes character string data composed of a plurality of words, which is a character string obtained as a result of speech recognition of a speech signal, and recognition results for each word constituting the entire character string data or character string data. And the reliability indicating the likelihood of. Then, the voice recognition unit 12 transmits the in-terminal voice recognition result to the command matching unit 16. Note that the speech recognition unit 12 cannot correctly recognize a speech when a word outside the vocabulary for in-terminal speech recognition that is not registered in the user dictionary 13 is included in the utterance content.

  The user dictionary 13 is a list in which words to be recognized are registered and listed. The user dictionary 13 may include words unique to the user of the terminal device 1 in addition to commonly used words. For example, a name registered in the phone book of the terminal device 1 and a content name such as music and video stored in the terminal device 1 are registered in the user dictionary 13 as words. In addition, since proper names such as place names, station names, product names, restaurant names, and application names are diverse, they may not be registered in the user dictionary 13.

  The voice transmission unit 14 functions as a voice transmission unit that transmits a voice signal to the voice recognition server 2. Specifically, the voice transmission unit 14 receives the voice signal transmitted by the voice input unit 11 and transmits the received voice signal to the voice recognition server 2 via the network NW. At this time, the audio transmission unit 14 transmits the audio signal after being uncompressed or compressed. The voice transmission unit 14 may temporarily store the voice signal received from the voice input unit 11 and transmit the voice signal to the voice recognition server 2 according to an instruction from the command verification unit 16 described later. In addition, the voice transmission unit 14 may transmit the voice signal received from the voice input unit 11 to the voice recognition server 2 without waiting for an instruction from the command matching unit 16.

  The server speech recognition result receiving unit 15 functions as a server speech recognition result receiving unit that receives a server speech recognition result that is a speech recognition result for a speech signal by the speech recognition server 2. The server voice recognition result receiving unit 15 receives a server voice recognition result from the voice recognition server 2 via the network NW. Then, the server voice recognition result receiving unit 15 transmits the received server voice recognition result to the command matching unit 16.

  The command collation unit 16 collates the speech recognition result in the terminal with a command dictionary 17 to be described later, determines which of the speech recognition results in the terminal and the server speech recognition result is to be used, and determines the determined speech It functions as a command collating means for specifying a command indicated by the voice signal based on the recognition result. More specifically, the command matching unit 16 first compares the in-terminal speech recognition result with the command dictionary 17 and calculates a certainty factor indicating the possibility of corresponding to the command. A method for calculating the certainty factor will be described later.

  When the certainty factor is equal to or greater than a predetermined threshold value, the command matching unit 16 determines the use of the in-terminal speech recognition result, and specifies a command having a certainty factor equal to or greater than the threshold value as a command indicated by the voice signal. On the other hand, when the certainty factor is less than the threshold value, that is, when the in-terminal speech recognition result cannot be received as a command, the command matching unit 16 determines the use of the server speech recognition result and receives it from the speech input unit 11 The voice transmitting unit 14 is instructed to transmit the voice signal to the voice recognition server 2. The threshold value is a fixed value set in advance in the command verification unit 16 and is set to 2.5, for example.

  When the command with the certainty level equal to or greater than the threshold is a command for instructing the execution of the in-terminal function, the command matching unit 16 determines the use of the in-terminal speech recognition result. When the command is a command other than a command for instructing execution of the in-terminal function, the use of the server speech recognition result may be determined. Here, the in-terminal function is a function provided in the terminal device 1 (for example, a telephone function, a mail function, a camera function, a schedule function, etc.), and without using other devices on the network NW, the terminal device This means a function that can be executed within 1. The command verification unit 16 may determine whether the function is an in-terminal function based on, for example, a keyword list described later. In this case, the command matching unit 16 can execute the command for instructing the execution of the in-terminal function at high speed using the in-terminal speech recognition result, and reliably use the server speech recognition result for other commands. Can be executed.

  The command dictionary 17 is a list of commands that can be used in the terminal device 1. The command dictionary 17 may include a command for instructing execution of an external function using an external server or the like in addition to a command for instructing execution of an in-terminal function. Commands for instructing execution of the in-terminal function include, for example, making a call, starting a mail function, starting a camera, and starting an in-terminal application. Further, commands for instructing execution of the function outside the terminal include transfer search, restaurant search, shopping site search, weather forecast browsing, and application search.

  FIG. 3 is a diagram illustrating an example of a keyword list stored in the command dictionary 17. As shown in FIG. 3, “keyword”, “function”, “score”, and “in-terminal function” are stored in the keyword list in association with each other. The information stored in the “keyword” is information indicating a keyword having a certain relevance to each function, such as “phone”, “mail”, “read”, “buy”, and the like. The information stored in the “function” is information indicating a function that can be executed by the terminal device 1, for example, “phone”, “mail”, “camera”, “transfer search”, “shopping search”, and the like. . This function includes the function outside the terminal in addition to the function inside the terminal. The information stored in the “score” is a value (score) indicating the degree of relevance between the keyword associated with the information and the function, and a higher value is assigned as the relevance is higher. The score is information used for calculating the certainty factor as will be described later. The information stored in “in-terminal function” is information indicating whether or not the function associated with the information is an in-terminal function.

  For example, the keyword “phone” is associated with a telephone function, a score of 2.0, and “Yes” indicating an in-terminal function. That is, when the word “telephone” is included in the speech recognition result, it is highly likely that the command is a command for instructing execution of the telephone function. Assigned.

  The keyword indicating the person name is associated with the telephone function and the mail function. A score of 0.5 is assigned to the telephone function, and a score of 0.5 is assigned to the mail function. Thus, a plurality of different functions may be associated with one keyword. In this case, since there are a plurality of functions associated with one keyword, it is not possible to specify which function is instructed to execute if the keyword is included in the speech recognition result. Therefore, a small score may be assigned to keywords associated with a plurality of functions. Further, a score may not be assigned to a keyword associated with a plurality of functions. The keyword indicating the person name may include a name and a name registered in the telephone directory function of the terminal device 1. The keyword indicating the person name may be set based on additional information such as part of speech included in the speech recognition result, or a name registered in a fixed person name dictionary may be used.

  The command execution unit 18 functions as a command execution unit that executes the command specified by the command verification unit 16. Then, the command matching unit 16 outputs the result of executing the command to the output device 108, for example.

  Subsequently, the function of the voice recognition server 2 will be described with reference to FIG. The voice recognition server 2 functionally includes a voice receiver 21 (voice receiver), a server voice recognizer 22 (server voice recognizer), a large vocabulary dictionary 23 (server dictionary), and a server voice recognition result transmission. Unit 24 (server voice recognition result transmission means).

  The voice receiving unit 21 functions as a voice receiving unit that receives a voice signal transmitted from the terminal device 1. Specifically, the voice reception unit 21 receives a voice signal from the voice transmission unit 14 of the terminal device 1 via the network NW, and transmits the received voice signal to the server voice recognition unit 22.

  The server speech recognition unit 22 functions as a server speech recognition unit that recognizes the speech signal received by the speech reception unit 21 based on the large vocabulary dictionary 23. More specifically, the server speech recognition unit 22 refers to an acoustic model and language model stored in advance and a large vocabulary dictionary 23 described later, and acquires a server speech recognition result. The server speech recognition result includes character string data composed of a plurality of words, which is a character string resulting from speech recognition of the speech signal, and the reliability of each word constituting the character string data. The server voice recognition unit 22 can perform voice recognition with higher accuracy than the voice recognition unit 12.

  The large vocabulary dictionary 23 includes more vocabularies than the user dictionary 13. In the large vocabulary dictionary 23, a wide variety of words including proper names such as place names, station names, product names, restaurant names, and application names are registered and listed.

  The server speech recognition result transmitting unit 24 functions as a server speech recognition result transmitting unit that transmits the server speech recognition result to the terminal device 1. Specifically, the server voice recognition result transmitting unit 24 receives a server voice recognition result for the voice signal received by the voice receiving unit 21 from the server voice recognition unit 22, and receives the received server voice recognition result via the network NW. To the server voice recognition result receiving unit 15 of the terminal device 1.

  Next, a certainty factor calculation method will be described with reference to an example shown in FIG. FIG. 4 is a diagram for explaining a certainty factor calculation method based on a speech recognition result. As shown in FIG. 4, the user inputs the speech content (a) “call phone to Yamada-san” using the voice input device 106, and the voice recognition unit 12 performs the in-terminal voice recognition result (b ) Suppose that "Call Yamada-san" is acquired. The command matching unit 16 refers to the keyword list and determines whether or not each word in the in-terminal speech recognition result (b) matches the keyword registered in the keyword list. When the word included in the in-terminal speech recognition result (b) matches the keyword registered in the keyword list, the command matching unit 16 is assigned to the function associated with the keyword and the function. Get the score from the keyword list.

  In this case, since “Yamada” corresponds to a personal name, it is determined that the keyword matches the keyword registered in the keyword list. Then, based on the keyword list, the command verification unit 16 acquires a score of 0.5 for the telephone function and acquires a score of 0.5 for the mail function. Further, since “phone” matches the keyword registered in the keyword list, the command matching unit 16 obtains a score of 2.0 for the telephone function associated with the keyword “phone”. Further, since “call” matches the keyword registered in the keyword list, the command matching unit 16 obtains a score of 0.5 for the telephone function associated with the keyword “call”.

<First confidence calculation method>
Here, the command matching unit 16 calculates the total score acquired for each function for each function, and the maximum value among them is set as the certainty of the command instructing the execution of the function having the maximum value. . In the example of FIG. 4, since the sum of the telephone function scores is 0.5 + 2.0 + 0.5 = 3.0 and the sum of the mail function scores is 0.5, the command for instructing the execution of the telephone functions The certainty factor is calculated as 3.0.

<Second confidence calculation method>
The command verification unit 16 calculates the total score acquired for each function for each function, and determines the difference between the maximum value and the second largest value for the command to instruct the execution of the function having the maximum value. Degree. In the example of FIG. 4, since the total score of the telephone function is 3.0 and the total score of the mail function is 0.5, the certainty of the command for instructing execution of the telephone function is 3.0−. 0.5 = 2.5 is calculated.

Moreover, when the reliability of the speech recognition result for each word is obtained, the following third certainty factor calculation method or fourth certainty factor calculation method may be used.
<Third confidence calculation method>
The command matching unit 16 calculates a certainty factor based on the score acquired for each function and the reliability of the speech recognition result for each word. For example, for each word “call Yamada-san” recognized by the voice recognition unit 12, the reliability of “Yamada” is 0.9, the reliability of “san” is 0.8, and the reliability of “ni” Assume that the degree is 0.8, the reliability of “telephone” is 1.0, the reliability of “O” is 0.5, and the reliability of “Call” is 0.5. This reliability is included in the in-terminal speech recognition result acquired by the speech recognition unit 12. In this case, if the command matching unit 16 determines that the word of the in-terminal speech recognition result matches the keyword registered in the keyword list, the reliability of the word is added to the score assigned to the function associated with the keyword. Add Then, the command matching unit 16 calculates the total score including the reliability for each function, and sets the maximum value among them as the certainty of the command instructing the execution of the function having the maximum value. In the example of FIG. 4, the total score including the reliability for the telephone function is 0.5 + 0.9 + 2.0 + 1.0 + 0.5 + 0.5 = 5.4, and the total score including the reliability for the mail function is 0.00. Since 5 + 0.9 = 1.4, the certainty factor of the command for instructing execution of the telephone function is calculated as 5.4.

<Fourth certainty factor calculation method>
When the word in the terminal speech recognition result matches the keyword registered in the keyword list, the command matching unit 16 multiplies the score assigned to the keyword by the reliability of the word. Then, the command matching unit 16 calculates the sum of the scores multiplied by the reliability for each function, and sets the maximum value as the certainty of the command instructing the execution of the function having the maximum value. If the reliability of each word in the in-terminal speech recognition result is the same as that described in the third certainty factor calculation method, in the example of FIG. 4, the total score multiplied by the reliability for the telephone function is 0.5 ×. 0.9 + 2.0 × 1.0 + 0.5 × 0.5 = 2.7, and the total score multiplied by the reliability of the mail function is 0.5 × 0.9 = 0.45. The certainty factor of the command for instructing execution is calculated as 2.7.

  Next, a command determination execution process (voice recognition method) executed by the terminal device 1 will be described.

<First command determination execution process>
FIG. 5 is a flowchart illustrating an example of command determination execution processing of the terminal device 1. This process is started when the user of the terminal device 1 inputs a voice via the voice input device 106.

  The voice input unit 11 receives an input of a voice signal from the voice input device 106 (S01, voice input step). Then, the voice input unit 11 transmits the received voice signal to the voice recognition unit 12 and the voice transmission unit 14. Next, the voice recognition unit 12 receives the voice signal transmitted by the voice input unit 11, and recognizes the received voice signal with reference to the user dictionary 13 (S02, voice recognition step). The speech recognition unit 12 transmits to the command verification unit 16 the in-terminal speech recognition result including character string data that is a character string obtained as a result of speech recognition of the speech signal and reliability information of each word constituting the character string data.

  Next, the command collation unit 16 collates the in-terminal speech recognition result with the command dictionary 17 (S03, command collation step). Specifically, the command matching unit 16 refers to the keyword list stored in the command dictionary 17 and determines whether each word included in the in-terminal speech recognition result matches the keyword registered in the keyword list. Determine. When the word included in the in-terminal speech recognition result matches the keyword, the command matching unit 16 acquires the function associated with the keyword and the score assigned to the function from the keyword list. And the command collation part 16 calculates the reliability of a command with either of the above-mentioned reliability calculation methods.

  Next, the command verification unit 16 determines whether or not the in-terminal speech recognition result is accepted as a command (S04, command verification step). That is, the command matching unit 16 determines whether or not the certainty factor is greater than or equal to a threshold value. When it is determined that the in-terminal speech recognition result is accepted as a command, that is, when the certainty factor is determined to be greater than or equal to the threshold (S04; Yes), the command matching unit 16 determines that the accepted command is an in-terminal function. It is determined whether or not the command is for instructing the execution of (S05). This determination is performed based on, for example, information indicating whether the keyword list is an in-terminal function.

  When it is determined that the accepted command is a command for instructing execution of the in-terminal function (S05; Yes), the command matching unit 16 identifies the command as a command indicated by the voice signal, and executes the command. The unit 18 is instructed to execute the command. Subsequently, the command execution unit 18 executes the command specified by the command verification unit 16 (S06, command execution step). Then, the terminal device 1 ends the command determination execution process.

  On the other hand, if it is determined in S04 that the in-terminal speech recognition result is not accepted as a command, that is, if it is determined that the certainty factor is less than the threshold (S04; No), or in the determination of S05. When it is determined that the accepted command is a command other than a command for instructing execution of the in-terminal function (S05; No), the command matching unit 16 sends a voice input unit to the voice transmitting unit 14. 11 is instructed to transmit the voice signal received from the voice recognition server 2 to the voice recognition server 2. Then, the voice transmission unit 14 transmits a voice signal to the voice recognition server 2 (S07, voice transmission step).

  When the voice recognition server 2 receives the voice signal transmitted by the voice transmission unit 14, the voice recognition server 2 performs voice recognition on the voice signal and acquires a server voice recognition result. This speech recognition uses a large vocabulary dictionary 23 having a larger vocabulary than the user dictionary 13, and is therefore more accurate than in-terminal speech recognition performed by the speech recognition unit 12. Then, the voice recognition server 2 transmits the server voice recognition result to the terminal device 1.

  Thereafter, the server voice recognition result receiving unit 15 receives the server voice recognition result from the voice recognition server 2 (S08, server voice recognition result receiving step). Then, the server voice recognition result receiving unit 15 transmits the received server voice recognition result to the command matching unit 16. Next, the command collation unit 16 collates the server voice recognition result with the command dictionary 17 (S09). More specifically, the command matching unit 16 refers to the keyword list stored in the command dictionary 17 and determines whether each word included in the server speech recognition result matches the keyword registered in the keyword list. judge. When the word included in the server speech recognition result matches the keyword, the command matching unit 16 acquires the function associated with the keyword and the score assigned to the function from the keyword list. And the command collation part 16 calculates the reliability of a command with the above-mentioned reliability calculation method.

  Next, the command verification unit 16 determines whether or not the server speech recognition result is accepted as a command (S10). That is, the command matching unit 16 determines whether or not the certainty factor is greater than or equal to a threshold value. When it is determined that the certainty factor is equal to or greater than the threshold, that is, when it is determined that the server speech recognition result is accepted as a command (S10; Yes), the command matching unit 16 uses the command indicated by the speech signal. The command execution unit 18 is instructed to execute the command. Subsequently, the command execution unit 18 executes the command specified by the command verification unit 16 (S06, command execution step). Then, the terminal device 1 ends the command determination execution process. Since the server speech recognition is more accurate than the intra-terminal speech recognition, the threshold value in the determination in S10 may be smaller than the threshold value in the determination in S04.

  On the other hand, if it is determined in S10 that the certainty factor is less than the threshold value, that is, if it is determined that the server speech recognition result has not been accepted as a command (S10; No), the terminal device 1 determines the command determination. The execution process is terminated. At this time, the terminal device 1 may display a message or the like for prompting re-input of voice on the output device 108.

<Second command discrimination execution process>
FIG. 6 is a flowchart illustrating another example of the command determination execution process of the terminal device 1. This process is started when the user of the terminal device 1 inputs a voice via the voice input device 106. In addition, since each step of S21-S29 in this process is the same as each step of S01-S09 of FIG. 5, description of S21-S29 is abbreviate | omitted.

  After the process of S29, the command collation unit 16 collates the in-terminal speech recognition result with the command dictionary 17 again (S30). Then, the command matching unit 16 compares the certainty factor of the command calculated in S29 with the certainty factor of the command calculated in S30, and selects the command having the highest certainty factor (S31). Then, the command verification unit 16 specifies the selected command as a command indicated by the voice signal, and instructs the command execution unit 18 to execute the command. The command execution unit 18 determines whether there is a command specified by the command verification unit 16 (S32). When it is determined that a command exists (S32; Yes), the command execution unit 18 executes the command (S26, command execution step). Then, the terminal device 1 ends the command determination execution process. On the other hand, when it is determined in S32 that the command does not exist (S32; No), the command execution unit 18 does not execute the command. Then, the terminal device 1 ends the command determination execution process. At this time, the terminal device 1 may display a message or the like for prompting re-input of voice on the output device 108.

  In S29, the command collation unit 16 may collate the server speech recognition result only with a command group that does not correspond to the in-terminal function, and in S30, the command speech recognition result may be compared with the command group that corresponds to the in-terminal function. You may only collate. In S31, the command matching unit 16 selects the command with the highest certainty factor, but may select the command with the highest certainty factor among the commands having the certainty factor equal to or higher than the threshold value. In this case, it is preferable that the command collation unit 16 collates with a threshold serving as a reference for accepting a command being smaller than the threshold in S04.

<Third command discrimination execution process>
FIG. 7 is a flowchart showing another example of command determination execution processing of the terminal device 1. This process is started when the user of the terminal device 1 inputs a voice via the voice input device 106. In addition, since each step of S41-S47 in this process is the same as each step of S01-S07 of FIG. 5, description of S41-S47 is abbreviate | omitted.

  In the voice recognition server 2, when the voice reception unit 21 receives the voice signal transmitted in S 47, the server voice recognition unit 22 refers to the large vocabulary dictionary 23 and recognizes the voice signal received by the voice reception unit 21. To obtain the server speech recognition result. Then, the server speech recognition unit 22 further collates the server speech recognition result with a command dictionary (not shown) provided in the speech recognition server 2. More specifically, the server speech recognition unit 22 refers to the keyword list stored in the command dictionary, and determines whether each of the words included in the server speech recognition result matches the keyword registered in the keyword list. judge.

  The keyword list may include keywords corresponding to more commands than the keyword list stored in the command dictionary 17. When the word included in the server speech recognition result matches the keyword, the server speech recognition unit 22 acquires the function associated with the keyword and the score assigned to the function from the keyword list. Then, the server speech recognition unit 22 calculates the certainty factor of the command by any one of the above certainty factor calculation methods.

  Next, the server voice recognition unit 22 determines whether the certainty factor is equal to or higher than a threshold value. When it is determined that the certainty factor is equal to or greater than the threshold value, that is, when it is determined that the server voice recognition result is accepted as a command, the server voice recognition result transmission unit 24 of the voice recognition server 2 has a certainty factor equal to or greater than the threshold value. The terminal device 1 is instructed to execute the command. Then, the server voice recognition result receiving unit 15 receives an instruction from the voice recognition server 2 (S48), and transmits the instruction to the command execution unit 18. Subsequently, the command execution unit 18 executes the instructed command (S46, command execution step). Then, the terminal device 1 ends the command determination execution process.

  In the first to third command determination execution processes, the voice transmission to the voice recognition server 2 (S07, S27, or S47, voice transmission step) is performed in the terminal voice recognition (S02, S22, or S42, voice recognition step). ) And the command verification of the in-terminal speech recognition result (S03, S23 or S43, command verification step), the voice transmission to the voice recognition server 2 is performed at an arbitrary timing after the voice input. Prior to in-terminal speech recognition or command verification of in-terminal speech recognition results, or in parallel with these steps. Thereby, the delay time until receiving the server speech recognition result can be shortened.

  Next, the processing will be specifically described with reference to FIGS. 4, 8, and 9 until a command is specified and executed based on the input utterance content.

  As described above, in the example illustrated in FIG. 4, first, the user inputs the utterance content (a) “call phone call to Yamada-san” via the voice input device 106. The voice input unit 11 receives a voice signal corresponding to the utterance content (a), and transmits the voice signal to the voice recognition unit 12 and the voice transmission unit 14. Next, the speech recognition unit 12 performs in-terminal speech recognition with reference to the user dictionary 13 and obtains the in-terminal speech recognition result (b) “call Yamada-san”. The speech recognition unit 12 transmits the in-terminal speech recognition result (b) to the command verification unit 16.

  Next, the command collation unit 16 collates the in-terminal speech recognition result (b) with the command dictionary 17. The command matching unit 16 determines that “Yamada” is a person name, and adds a score of 0.5 to each of the telephone function and the mail function associated with the keyword “<person name>” in the keyword list. Further, since “phone” matches the keyword “phone” in the keyword list, the command matching unit 16 adds a score of 2.0 to the telephone function associated with the keyword “phone”. Further, the command matching unit 16 adds a score of 0.5 to the telephone function associated with the keyword “call” because “call” matches the keyword “call” in the keyword list. Then, the command matching unit 16 calculates a certainty factor based on the matching result. Here, the certainty factor is calculated by the above-described second certainty factor calculation method, and the threshold value is set to 2.0. In this case, the telephone function has the highest score, and its certainty is 3.0−0.5 = 2.5.

  Then, the command verification unit 16 compares the certainty factor with a threshold value and determines whether or not to accept a command for executing the telephone function. Since the certainty factor is greater than or equal to the threshold, the command matching unit 16 accepts a command for executing the telephone function. Next, the command verification unit 16 determines whether or not the telephone function is an in-terminal function. Since the telephone function is an in-terminal function, the command verification unit 16 specifies the command indicated by the generated content (a) as a command for executing the telephone function, and instructs the command execution unit 18 to execute the command. . Then, the command execution unit 18 executes a command for executing the telephone function.

  FIG. 8 is a diagram illustrating an example for explaining command determination execution processing in the speech recognition system 10. First, the utterance content (a) “Turn over” is input via the voice input device 106 by the user. The voice input unit 11 receives a voice signal corresponding to the utterance content (a), and transmits the voice signal to the voice recognition unit 12 and the voice transmission unit 14. Next, the speech recognition unit 12 refers to the user dictionary 13 to perform in-terminal speech recognition, and obtains the in-terminal speech recognition result (b) “Buo Sugao”. The speech recognition unit 12 transmits the in-terminal speech recognition result (b) to the command verification unit 16.

  Next, the command collation unit 16 collates the in-terminal speech recognition result (b) with the command dictionary 17. The command matching unit 16 determines that “Tatsuo” is a person name, and adds a score of 0.5 to each of the telephone function and the mail function associated with the keyword “<person name>” in the keyword list. Furthermore, since “buy” matches the keyword in the keyword list, the command matching unit 16 adds a score of 2.0 to the shopping function associated with the keyword “buy”. Then, the command matching unit 16 calculates a certainty factor based on the matching result. Here, the certainty factor is calculated by the above-described second certainty factor calculation method, and the threshold value is set to 2.0. In this case, the shopping function has the highest score, and its certainty is 2.0−0.5 = 1.5.

  And the command collation part 16 compares a certainty factor with a threshold value, and determines whether the command for performing a shopping function is received. Since the certainty factor is smaller than the threshold value, the command matching unit 16 does not accept a command for executing the shopping function. Next, the command verification unit 16 instructs the voice transmission unit 14 to transmit a voice signal corresponding to the utterance content (a) to the voice recognition server 2. Then, the voice transmission unit 14 transmits a voice signal corresponding to the utterance content (a) to the voice recognition server 2. Thereafter, the server voice recognition result receiving unit 15 receives the server voice recognition result (c) “buy an umbrella” from the voice recognition server 2. Then, the server voice recognition result receiving unit 15 transmits the server voice recognition result (c) to the command matching unit 16.

  Next, the command collation unit 16 collates the server voice recognition result (c) with the command dictionary 17. The command matching unit 16 adds a score of 2.0 to the shopping function associated with the keyword “buy” because “buy” matches the keyword in the keyword list. The command matching unit 16 calculates a certainty factor based on the matching result. In this case, the shopping function has the highest score, and its certainty is 2.0. And the command collation part 16 compares a certainty factor with a threshold value, and determines whether the command for performing a shopping function is received.

  Here, the threshold value may be the same as the value used for determining whether or not the in-terminal speech recognition result is accepted as a command, but is preferably smaller. Here, the threshold is set to 0.5. Since the certainty factor is greater than or equal to the threshold, the command matching unit 16 accepts a command for executing the shopping function. Then, the command verification unit 16 specifies the command indicated by the utterance content (a) as a command for executing the shopping function, and instructs the command execution unit 18 to execute the command. Then, the command execution unit 18 executes a command for executing the shopping function.

  If the intra-terminal speech recognition result (b) is “buy an umbrella” and the threshold used to determine whether the intra-terminal speech recognition result (b) is accepted as a command is 1.5, Since the certainty factor based on the collation result of the in-terminal speech recognition result (b) is equal to or greater than the threshold value, the command collation unit 16 accepts a command for executing the shopping function. However, since the shopping function does not correspond to the in-terminal function, in this case as well, the command matching unit 16 instructs the voice transmitting unit 14 to transmit the voice signal corresponding to the utterance content (a) to the voice recognition server 2. become.

  FIG. 9 is a diagram illustrating another example for explaining command determination execution processing in the speech recognition system 10. First, an utterance content (a) “call to Yamada-san” is input via the voice input device 106 by the user. The voice input unit 11 receives a voice signal corresponding to the utterance content (a), and transmits the voice signal to the voice recognition unit 12 and the voice transmission unit 14. Next, the speech recognition unit 12 refers to the user dictionary 13 to perform in-terminal speech recognition, and acquires the in-terminal speech recognition result (b) “Call to Mr. Yamada”. The speech recognition unit 12 transmits the in-terminal speech recognition result (b) to the command verification unit 16.

  Next, the command collation unit 16 collates the in-terminal speech recognition result (b) with the command dictionary 17. The command matching unit 16 determines that “Yamada” is a person name, and adds a score of 0.5 to each of the telephone function and the mail function associated with the keyword “<person name>” in the keyword list. Further, the command matching unit 16 adds a score of 0.5 to the telephone function associated with the keyword “calling” because “calling” matches the keyword in the keyword list. Then, the command matching unit 16 calculates a certainty factor based on the matching result. Here, the certainty factor is calculated by the second certainty factor calculation method, and the threshold value is set to 2.0. In this case, the telephone function has the highest score, and its certainty is 1.0−0.5 = 0.5.

  Then, the command verification unit 16 compares the certainty factor with a threshold value and determines whether or not to accept a command for executing the telephone function. Since the certainty factor is smaller than the threshold value, the command matching unit 16 does not accept a command for executing the telephone function. Next, the command verification unit 16 instructs the voice transmission unit 14 to transmit a voice signal corresponding to the utterance content (a) to the voice recognition server 2. Then, the voice transmission unit 14 transmits a voice signal corresponding to the utterance content (a) to the voice recognition server 2. Thereafter, the server voice recognition result receiving unit 15 receives the server voice recognition result (c) “Call to Mr. Yamada” from the voice recognition server 2. Then, the server voice recognition result receiving unit 15 transmits the server voice recognition result (c) to the command matching unit 16.

  Next, the command collation unit 16 collates the server voice recognition result (c) with the command dictionary 17. The command matching unit 16 determines that “Yamada” is a person name, and adds a score of 0.5 to each of the telephone function and the mail function associated with the keyword “<person name>” in the keyword list. Further, the command matching unit 16 adds a score of 0.5 to the telephone function associated with the keyword “calling” because “calling” matches the keyword in the keyword list. Then, the command matching unit 16 calculates a certainty factor based on the matching result. In this case, the telephone function has the highest score, and its certainty is 1.0−0.5 = 0.5. Then, the command verification unit 16 compares the certainty factor with a threshold value and determines whether or not to accept a command for executing the telephone function.

  Here, the threshold is set to 0.5, which is smaller than the value used for determining whether or not the in-terminal speech recognition result is accepted as a command. The command verification unit 16 accepts a command for executing the telephone function because the certainty factor is equal to or greater than the threshold value. Then, the command verification unit 16 specifies the command indicated by the generated content (a) as a command for executing the telephone function, and instructs the command execution unit 18 to execute the command. Then, the command execution unit 18 executes a command for executing the telephone function.

  As described above, for example, when the user utters a command for executing the in-terminal function, the voice is correctly recognized by the in-terminal voice recognition, and the command is executed without performing the server voice recognition. On the other hand, when a user utters a command intended to acquire information from a network, proper names such as place names, station names, product names, restaurant names, and application names may be included in the commands. In such a case, since the voice recognition cannot be performed correctly by the in-terminal voice recognition, the command is executed after being reliably recognized by the server voice recognition.

  Next, an example in which the command determination execution process of the terminal device 1 is applied to the function / application determination call process will be described. This process is a process of calling a function (telephone, mail, scheduler, etc.) in the terminal device 1 or an application installed in the terminal device 1 when the user speaks to the terminal device 1. FIG. 10 and FIG. 11 are flowcharts for explaining an example of the function / application discrimination call processing in the speech recognition system 10. This process is started when the user of the terminal device 1 inputs a voice via the voice input device 106. In addition, since each step of S51-S52 in this process is the same as each step of S01-S02 of FIG. 5, description of S51-S52 is abbreviate | omitted.

  After the process of S52, the speech recognition unit 12 transmits the in-terminal speech recognition result acquired in S52 to the command verification unit 16. Next, the command collation unit 16 collates the in-terminal speech recognition result with the command dictionary 17 (S53). More specifically, the command matching unit 16 refers to the keyword list stored in the command dictionary 17 and determines whether the in-terminal speech recognition result matches the keyword registered in the keyword list (S54). . In the keyword list, a plurality of applications are registered in advance in addition to commands that can be used in the terminal device 1. In the keyword list, at least information indicating a keyword and information indicating a function or an application are stored in association with each other. A function name or application name is registered as this keyword. Apps registered in the keyword list are not limited to apps installed in the terminal device 1, but include apps that can be installed in the terminal device 1, such as popular apps.

  If it is determined in S54 that the in-terminal speech recognition result matches the keyword (S54; Yes), the command matching unit 16 determines that the function or application call is associated with the keyword, and the output device 108 “Are you sure you want to use (function name or application name)? Yes / other candidates” is displayed (S55). Then, the command verification unit 16 determines whether or not “Yes” is selected by the user (S56). When it is determined that the “other candidate” is selected by the user (S56; No), the command verification unit 16 transmits the voice signal received from the voice input unit 11 to the voice recognition server 2 to the voice transmission unit 14. Instruct them to do so. Also, in the determination of S54, when the in-terminal speech recognition result does not match the keyword (S54; No), the command matching unit 16 also sends the voice signal received from the voice input unit 11 to the voice transmission unit 14. Is transmitted to the voice recognition server 2.

  Then, the voice transmission unit 14 transmits a voice signal to the voice recognition server 2 (S57). When receiving the voice signal transmitted in S57, the voice recognition server 2 performs voice recognition and transmits the server voice recognition result to the terminal device 1. Thereafter, the server voice recognition result receiving unit 15 receives the server voice recognition result from the voice recognition server 2 (S58). Then, the server voice recognition result receiving unit 15 transmits the received server voice recognition result to the command matching unit 16. Next, the command collation unit 16 collates the server voice recognition result with the command dictionary 17 (S59). Specifically, the command matching unit 16 refers to the keyword list stored in the command dictionary 17 and determines whether or not the server speech recognition result matches the keyword registered in the keyword list. Further, the command collation unit 16 collates the in-terminal speech recognition result with the command dictionary 17 again (S60).

  Then, the command collation unit 16 acquires a list of keywords (function names or application names) that match the collation results in S59 and S60, the server speech recognition result, and the in-terminal speech recognition result. Then, the command matching unit 16 displays the acquired list of function names or application names on the output device 108 as candidates (S61). Thereafter, the command matching unit 16 determines whether any candidate has been selected by the user (S62). When it is determined that no candidate has been selected (S62; No), the terminal device 1 ends the function / application determination call processing. At this time, the terminal device 1 may display a message or the like for prompting re-input of voice on the output device 108.

  On the other hand, if it is determined in S62 that a candidate is selected (S62; Yes), or if it is determined in S56 that “Yes” is selected by the user (S56; Yes), command verification is performed. The unit 16 determines whether the selected candidate is a function or an application with reference to the keyword list stored in the command dictionary 17 (S63). When it is determined that the selected candidate is a function (S63; function), the command matching unit 16 specifies a command for executing the function as a command indicated by the audio signal, and executes the command. The execution unit 18 is instructed. Then, the command execution unit 18 executes a command for executing the selected function (S64), and the terminal device 1 ends the function / application determination calling process.

  On the other hand, if it is determined in S63 that the selected candidate is an application (S63; application), the command matching unit 16 determines whether or not the application is installed in the terminal device 1 ( S65). If it is determined that the selected application is installed (S65; Yes), the command collation unit 16 identifies the application as an application indicated by the audio signal, and starts the application to be executed by the command execution unit 18. To instruct. Then, the command execution unit 18 activates the selected application (S66), and the terminal device 1 ends the function / application determination call process. On the other hand, if it is determined in S65 that the selected application is not installed (S65; No), the command matching unit 16 instructs the output device 108 to “Do you want to search for an application? Yes / No. Is displayed (S67).

  Then, the command matching unit 16 determines whether “Yes” is selected by the user (S68). When it is determined that “Yes” is selected by the user (S68; Yes), the command verification unit 16 searches the application market, which is an application search site on the Internet, via the communication module 105. (S69). Then, the command matching unit 16 displays the search result on the output device 108 so that the user can install the application. Then, the terminal device 1 ends the function / application discrimination call process. On the other hand, in the determination of S68, when it is determined that “No” is selected by the user (S68; No), the terminal device 1 ends the function / application determination calling process.

  Similar to the above-described command determination execution process, the keyword list includes information indicating a keyword, information indicating a function name or an application name, information indicating a score, and information indicating whether the function is an application. , May be stored in association with each other. In this case, the collation of the in-terminal speech recognition result and the server speech recognition result is performed by comparing the word included in the speech recognition result with the keyword registered in the keyword list, as in the above-described command determination execution process. It may be performed by calculating the certainty factor by any one of the certainty factor calculation methods.

  Next, a process for identifying an application based on the input utterance content will be specifically described with reference to FIG. FIG. 12 is a diagram illustrating an example for explaining the function / application discrimination call processing in the speech recognition system 10.

  First, the utterance content (a) “Flat Todo” is input by the user via the voice input device 106. The voice input unit 11 receives a voice signal corresponding to the utterance content (a), and transmits the voice signal to the voice recognition unit 12 and the voice transmission unit 14. Next, the speech recognition unit 12 performs in-terminal speech recognition with reference to the user dictionary 13 and acquires the in-terminal speech recognition result (b) “bath thread”. The speech recognition unit 12 transmits the in-terminal speech recognition result (b) to the command verification unit 16.

  Next, the command collation unit 16 collates the in-terminal speech recognition result (b) with the command dictionary 17. Specifically, the command matching unit 16 determines whether or not the in-terminal speech recognition result (b) matches the keyword (function name and application name) registered in the keyword list. Since the in-terminal speech recognition result does not match the keyword, the command matching unit 16 instructs the speech transmitting unit 14 to transmit the speech signal corresponding to the utterance content (a) to the speech recognition server 2. Then, the voice transmission unit 14 transmits a voice signal corresponding to the utterance content (a) to the voice recognition server 2.

  In the voice recognition server 2, voice recognition is performed on the received voice signal using the large vocabulary dictionary 23 in which the name of the application in the city is registered. Thereafter, the server speech recognition result receiving unit 15 receives the server speech recognition result (c) “flood it” from the speech recognition server 2. Then, the server voice recognition result receiving unit 15 transmits the server voice recognition result (c) to the command matching unit 16.

  Next, the command collation unit 16 collates the server voice recognition result (c) with the command dictionary 17. Specifically, the command matching unit 16 determines whether or not the server speech recognition result (c) matches a keyword (function name and application name) registered in the keyword list. As a result, it is determined that the server speech recognition result (c) matches the keyword “Flood-It”. Then, the command matching unit 16 displays “Flood-It” as a candidate on the output device 108. Thereafter, the command matching unit 16 determines whether or not a candidate has been selected by the user, and determines that “Flood-It” has been selected.

  Subsequently, the command matching unit 16 determines whether the selected candidate “Flood-It” is a function or an application with reference to the keyword list. Since the keyword “Flood-It” is registered as an application in the keyword list, the command matching unit 16 determines that the application is an application. Then, the command verification unit 16 determines whether or not the application “Flood-It” is installed in the terminal device 1. As a result of the determination, since the application “Flood-It” is not installed in the terminal device 1, the command matching unit 16 displays on the output device 108 whether or not to perform an application search.

  If the user selects to perform an application search, the command matching unit 16 searches the application market for the application “Flood-It” via the communication module 105 and displays the search result on the output device 108. To do. Then, when installation of the application is instructed by the user, the application “Flood-It” is installed in the terminal device 1. As described above, the user can install a desired application on the terminal device 1 by simple operations such as utterance of an application name, confirmation of a speech recognition result, an instruction for searching for an application, and an instruction for installing an application.

  Next, the effect of the terminal device 1 is demonstrated. In the terminal device 1, the voice input unit 11 receives an input of a voice signal, and the voice recognition unit 12 performs voice recognition on the voice signal. Further, the voice transmission unit 14 transmits a voice signal to the voice recognition server 2, and the server voice recognition result reception unit 15 receives the server voice recognition result. Then, the command collation unit 16 collates the in-terminal speech recognition result with the command dictionary 17 to determine which of the in-terminal speech recognition result and the server speech recognition result to use, and the determined speech A command indicated by the voice signal is specified based on the recognition result. Then, the command execution unit 18 executes the command specified by the command verification unit 16.

  Thus, for example, if the in-terminal speech recognition result is accepted as a command, the command is executed using the in-terminal speech recognition result, and if the in-terminal speech recognition result cannot be accepted as a command, the server speech The recognition result can be used. In other words, when a command that can be recognized by the vocabulary in the terminal is input by voice, a command that cannot be recognized by the vocabulary in the terminal can be input by using the result of speech recognition in the terminal. In this case, the command inputted by voice can be surely recognized and executed by using the server voice recognition result.

  In addition, the command matching unit 16 compares the in-terminal speech recognition result with the command dictionary 17 to calculate a certainty factor, and determines the use of the in-terminal speech recognition result when the certainty factor is equal to or greater than a predetermined threshold. The command having the certainty level equal to or higher than the threshold is specified as the command indicated by the voice signal.

  Thereby, although the reliability of the in-terminal speech recognition result is lowered due to ambient noise or the like, the in-terminal speech recognition result can be used when the speech is recognized correctly. As a result, a command that can be recognized by the vocabulary in the terminal can be executed at high speed even if the reliability of the speech recognition result in the terminal is low.

  The command dictionary 17 stores a keyword list in which a plurality of keywords and scores associated with the plurality of keywords are registered for each of the plurality of commands. For each word included in the recognition result, determine whether it corresponds to one of a plurality of keywords registered in the keyword list, and calculate the certainty factor based on the command and score associated with the corresponding keyword To do. Thereby, it can be determined whether or not the command can be recognized by the vocabulary in the terminal, and the command indicated by the voice signal can be recognized more reliably.

  Further, the command matching unit 16 determines whether each of the words included in the in-terminal speech recognition result corresponds to any of a plurality of keywords registered in the keyword list, and is associated with the corresponding keyword. It is preferable to calculate the certainty factor based on the command and score and the reliability of speech recognition of the word. Thereby, it can be determined whether or not the command can be recognized by the vocabulary in the terminal, and the command indicated by the voice signal can be recognized more reliably.

  In addition, when the command with the certainty level equal to or greater than the threshold is a command for instructing the execution of the in-terminal function, the command matching unit 16 determines the use of the in-terminal speech recognition result, and the command with the certainty level equal to or greater than the threshold value. However, if it is not a command for instructing execution of the in-terminal function, the use of the server speech recognition result is determined. As a result, commands for instructing execution of in-terminal functions can be executed at high speed using the in-terminal speech recognition results, and other commands can be reliably executed using the server speech recognition results. .

  The voice transmission unit 14 transmits a voice signal to the voice recognition server 2 before the voice recognition unit 12 obtains the in-terminal voice recognition result. Thereby, the server speech recognition result can be received earlier. For this reason, when it is determined to use the server speech recognition result, the command can be executed at high speed.

  The server speech recognition result receiving unit 15 discards the server speech recognition result when the command collating unit 16 receives the server speech recognition result after the command is specified based on the in-terminal speech recognition result. Thereby, a command can be specified based on the in-terminal speech recognition result without waiting for the reception of the server speech recognition result. For this reason, it becomes possible to execute a command recognizable by the vocabulary in the terminal at high speed.

  As described above, the terminal device 1 can use the in-terminal voice recognition and the network type voice recognition using the voice recognition server 2 in cooperation. This intra-terminal speech recognition can respond at high speed because communication via the network NW is unnecessary, but there are cases where accurate speech recognition cannot be performed because there are few vocabularies registered in the user dictionary 13. On the other hand, in the network type speech recognition, since the vocabulary registered in the large vocabulary dictionary 23 is larger than that in the user dictionary 13, the accuracy of speech recognition is higher than in-terminal speech recognition, but communication is performed via the network NW. The response may be delayed due to a communication delay or the like. Therefore, when a voice command recognizable in the vocabulary to be recognized in the terminal device 1 (including user-dependent vocabulary) is input to the terminal device 1, a high-speed response is enabled by in-terminal speech recognition, When a voice command that cannot be recognized by the recognition target vocabulary in the terminal device 1 is input, a large vocabulary and high-accuracy voice recognition is enabled by network type voice recognition using the voice recognition server 2.

  In addition, in this embodiment, although the terminal device 1 was illustrated as an apparatus, it is not restricted to this, For example, you may comprise as a program module for performing each function in the terminal device 1. FIG. That is, a voice input module corresponding to the voice input unit 11, a voice recognition module corresponding to the voice recognition unit 12, a voice transmission module corresponding to the voice transmission unit 14, and a server voice recognition result reception corresponding to the server voice recognition result reception unit 15. A speech recognition program including a module, a command verification module corresponding to the command verification unit 16, and a command execution module corresponding to the command execution unit 18, and the above-described program is read by a computer system such as a portable terminal. Functions equivalent to those of the terminal device 1 can be realized. The above speech recognition program is provided by being stored in a storage medium such as a flexible disk, CD-ROM, DVD or ROM, or a semiconductor memory, for example. Further, the above-described speech recognition program may be provided via a network as a computer data signal superimposed on a carrier wave.

  In addition, the voice transmission unit 14 may send an uncompressed voice waveform or a compressed voice waveform as a voice signal to the voice recognition server 2, or a feature amount used for voice recognition is sent to the voice recognition server 2. You may send it.

  In addition, after receiving the voice signal from the voice input unit 11, the voice transmission unit 14 may transmit the voice signal to the voice recognition server 2 before determining whether to use the in-terminal voice recognition result. Good. In this case, in parallel with the in-terminal speech recognition by the speech recognition unit 12 and the command verification of the in-terminal speech recognition result by the command verification unit 16, server speech recognition can be performed in the speech recognition server 2. It becomes possible to obtain it early. As a result, it is possible to reduce the processing time when the command collation unit 16 decides to use the server speech recognition result. At this time, when the command collation unit 16 acquires the server speech recognition result before the in-terminal speech recognition result, it is desirable to determine which speech recognition result to use after waiting for the in-terminal speech recognition result. However, the server speech recognition result may be used with priority.

  Further, the voice transmission unit 14 transmits a voice signal to the voice recognition server 2 before determining whether or not to use the in-terminal voice recognition result, and the server voice recognition result receiving unit 15 sends the voice recognition server 2 to the server. If the command matching unit 16 decides to use the in-terminal speech recognition result before receiving the speech recognition result, the command matching unit 16 performs speech recognition processing in the speech recognition server 2 on the speech transmitting unit 14. You may instruct | indicate to transmit the signal for canceling to the speech recognition server 2. FIG. At this time, the voice transmitting unit 14 may transmit a signal for canceling the voice recognition processing in the voice recognition server 2 to the voice recognition server 2 based on an instruction from the command matching unit 16. As described above, when a command that can be recognized by in-terminal speech recognition is input, it is determined that the in-terminal speech recognition result is used without waiting for the server speech recognition result from the speech recognition server 2. The command execution unit 18 can execute commands at high speed.

  In addition, the certainty threshold is not limited to a fixed value, and may be a value obtained by multiplying the number of words included in the in-terminal speech recognition result by a predetermined ratio (for example, 0.2). In this way, the threshold value can be dynamically changed according to the number of words, and the command can be specified more accurately.

  In addition, the command matching unit 16 is not limited to the case where the accepted command corresponds to an in-terminal function, for example, a function that uses only a limited word, that is, a function that does not require large vocabulary recognition by the speech recognition server 2. May correspond to the command.

  The command dictionary 17 may store a keyword list that associates keywords such as function names with functions and does not have a score. In this case, the command matching unit 16 determines whether or not the word or the partial character string included in the speech recognition result matches the keyword registered in the keyword list. A command for executing a function associated with a keyword may be specified as a command indicated by a voice signal.

  DESCRIPTION OF SYMBOLS 1 ... Terminal device, 2 ... Voice recognition server, 10 ... Voice recognition system, 11 ... Voice input part (voice input means), 12 ... Voice recognition part (voice recognition means), 13 ... User dictionary, 14 ... Voice transmission part ( Voice transmitting means), 15 ... server voice recognition result receiving section (server voice recognition result receiving means), 16 ... command matching section (command matching means), 17 ... command dictionary, 18 ... command execution section (command execution means), 21 ... voice receiver (voice receiver), 22 ... server voice recognizer (server voice recognizer), 23 ... large vocabulary dictionary (server dictionary), 24 ... server voice recognition result transmitter (server voice recognition result transmitter).

Claims (16)

An audio input means for receiving an input of an audio signal;
Voice recognition means for performing voice recognition on the voice signal received by the voice input means;
Voice transmitting means for transmitting the voice signal to a voice recognition server;
Server speech recognition result receiving means for receiving a server speech recognition result which is a speech recognition result for the speech signal by the speech recognition server;
A command dictionary in which multiple commands are registered;
Which speech recognition result is used, that is, the speech recognition result in the terminal, which is the speech recognition result recognized by the speech recognition means, is collated with the command dictionary and the speech recognition result in the terminal or the server speech recognition result is used. Command collating means for determining and identifying a command indicated by the voice signal based on the determined voice recognition result;
Command execution means for executing the command specified by the command verification means;
A terminal device comprising:   The command collating unit collates the in-terminal speech recognition result with the command dictionary, calculates a certainty factor indicating a possibility of corresponding to a command, and when the certainty factor is equal to or greater than a predetermined threshold, The terminal device according to claim 1, wherein use of an internal speech recognition result is determined, and the command having the certainty factor equal to or greater than the threshold is specified as a command indicated by the speech signal. The command dictionary stores a keyword list in which a plurality of keywords and scores associated with the plurality of keywords are registered for each of the plurality of commands.
The command matching unit determines whether each of the words included in the in-terminal speech recognition result corresponds to any of the plurality of keywords registered in the keyword list, and associates with the corresponding keyword. The terminal device according to claim 2, wherein the certainty factor is calculated based on the received command and score.   The command matching unit determines whether each of the words included in the in-terminal speech recognition result corresponds to any of the plurality of keywords registered in the keyword list, and associates with the corresponding keyword. The terminal device according to claim 3, wherein the certainty factor is calculated based on the received command and score and the reliability of voice recognition of the word.   When the command having the certainty factor equal to or greater than the threshold is a command for instructing execution of an in-terminal function, the command matching unit determines use of the intra-terminal speech recognition result, and the command equal to or greater than the threshold. The use of the server speech recognition result is determined when the command of certainty is a command other than a command for instructing execution of an in-terminal function. The terminal device described in 1.   6. The voice transmitting unit according to claim 1, wherein the voice signal is transmitted to the voice recognition server before the voice recognition unit obtains the intra-terminal voice recognition result. The terminal device described.   The server speech recognition result receiving unit discards the server speech recognition result when the command collating unit receives the server speech recognition result after a command is specified based on the in-terminal speech recognition result. The terminal device according to claim 6. A voice input module that accepts input of voice signals,
A voice recognition module for performing voice recognition on a voice signal received by the voice input module;
A voice transmission module for transmitting the voice signal to a voice recognition server;
A server speech recognition result receiving module for receiving a server speech recognition result that is a speech recognition result for the speech signal by the speech recognition server;
The in-terminal speech recognition result, which is the speech recognition result recognized by the speech recognition module, is checked against a command dictionary in which a plurality of commands are registered, and any one of the in-terminal speech recognition result and the server speech recognition result A command verification module that determines whether to use a recognition result and identifies a command indicated by the voice signal based on the determined voice recognition result;
A command execution module for executing the command specified by the command verification module;
A speech recognition program comprising:   The command verification module compares the in-terminal speech recognition result with the command dictionary, calculates a certainty factor indicating the possibility of corresponding to a command, and when the certainty factor is equal to or greater than a predetermined threshold, 9. The speech recognition program according to claim 8, wherein use of an internal speech recognition result is determined, and the command having the certainty level equal to or greater than the threshold is specified as a command indicated by the speech signal. The command dictionary stores a keyword list in which a plurality of keywords and a score associated with each of the plurality of keywords are registered for each of the plurality of commands.
The command verification module determines whether each of the words included in the in-terminal speech recognition result corresponds to any of the plurality of keywords registered in the keyword list, and associates with the corresponding keyword The speech recognition program according to claim 9, wherein the certainty factor is calculated based on the received command and score.   The command verification module determines whether each of the words included in the in-terminal speech recognition result corresponds to any of the plurality of keywords registered in the keyword list, and associates with the corresponding keyword The speech recognition program according to claim 10, wherein the certainty factor is calculated based on the received command and score and the reliability of speech recognition of the word.   The command verification module determines use of the intra-terminal speech recognition result when the command with the certainty level equal to or higher than the threshold is a command for instructing execution of an intra-terminal function, and the command verification module The use of the server speech recognition result is determined when the command of certainty is a command other than a command for instructing execution of an in-terminal function. The speech recognition program described in 1.   The voice transmission module transmits the voice signal to the voice recognition server before the voice recognition result is obtained by the voice recognition module. The described voice recognition program.   The server speech recognition result receiving module discards the server speech recognition result when receiving the server speech recognition result after a command is specified based on the in-terminal speech recognition result by the command verification module. The voice recognition program according to claim 13. An audio input step for receiving an input of an audio signal;
A voice recognition step for performing voice recognition on the voice signal received in the voice input step;
A voice transmission step of transmitting the voice signal to a voice recognition server;
A server speech recognition result receiving step for receiving a server speech recognition result which is a speech recognition result for the speech signal by the speech recognition server;
The in-terminal speech recognition result, which is the speech recognition result recognized in the speech recognition step, is collated with a command dictionary in which a plurality of commands are registered, and any one of the in-terminal speech recognition result and the server speech recognition result A command matching step for determining whether to use a recognition result, and for identifying a command indicated by the voice signal based on the determined voice recognition result;
A command execution step for executing the command specified in the command verification step;
A speech recognition method comprising: A speech recognition system including a terminal device and a speech recognition server,
The terminal device
An audio input means for receiving an input of an audio signal;
Voice recognition means for performing voice recognition on the voice signal received by the voice input means;
A command dictionary in which multiple commands are registered;
Voice transmitting means for transmitting the voice signal to the voice recognition server;
Server speech recognition result receiving means for receiving a server speech recognition result which is a speech recognition result for the speech signal by the speech recognition server;
Which speech recognition result is used, that is, the speech recognition result in the terminal, which is the speech recognition result recognized by the speech recognition means, is collated with the command dictionary and the speech recognition result in the terminal or the server speech recognition result is used. Command collating means for determining and identifying a command indicated by the voice signal based on the determined voice recognition result;
Command execution means for executing the command specified by the command verification means;
With
The voice recognition server
Voice receiving means for receiving the voice signal transmitted from the voice transmitting means;
A server dictionary containing more vocabulary than the terminal device;
Server voice recognition means for recognizing the voice signal received by the voice reception means based on the server dictionary;
Server speech recognition result transmitting means for transmitting the server speech recognition result to the terminal device;
A speech recognition system comprising:

Images