JP5558284B2 - Speech recognition system, speech recognition method, and speech recognition program - Google Patents

Description

  The present invention relates to a communication terminal, a voice recognition method, and a voice recognition program.

  The voice recognition process may be performed in the client terminal or on the server side. When performed in a client terminal, the terminal is often dedicated to the user. Therefore, as user-specific information in the terminal, user dictionary, user input voice and call voice, acoustic training results, etc. It is easy to personally customize language models and acoustic models. Accordingly, user-adaptive speech recognition is possible, but there are disadvantages such as a limited amount of memory and computing resources, and a restriction on the number of vocabularies and the range of hypothesis search.

  On the other hand, when it is performed on the server side, there are merits that a large amount of memory and computing resources can be used compared to in-terminal speech recognition, and speech recognition with high vocabulary and high accuracy is possible. However, since the server is shared by many users, it learns each user's language model or acoustic model, and loads the language model or acoustic model customized for each user immediately and quickly according to access. Is expensive. Therefore, there is a demerit that it is difficult to customize the language model or the acoustic model for each user in the speech recognition on the server side.

  Therefore, there is a problem of realizing a speech recognition process that combines the advantages of both, a large vocabulary and highly accurate speech recognition process, and customization of each language model or acoustic model for each user. One attempt to solve the problem is shown. In Patent Document 1, the server returns the time information of the word boundary in the recognition result to the terminal, and the terminal refers to the time information and performs re-recognition using a dictionary owned by itself. In particular, we aim to improve recognition accuracy by re-recognizing only words that are determined as unknown words or proper nouns.

JP 2010-85536 A

  However, in such a technique of Patent Document 1, if the word boundary is not correctly recognized by the server side recognition, the terminal side will refer to the incorrect time information, and the correct recognition result even after re-recognition at the terminal May not be obtained. Also, if the word to be recognized on the server side is a word outside the vocabulary, it may not be possible to correctly determine unknown words or proper nouns, and correct recognition results may not be obtained even after re-recognition at the terminal There is.

  Therefore, the present invention has been made in view of the above, and is a communication terminal capable of realizing speech recognition processing that achieves both large vocabulary and high-accuracy speech recognition processing and customization of each language model or acoustic model for each user. An object is to provide a speech recognition method and a speech recognition program.

  In order to solve the above problems, a communication terminal according to the present invention stores a voice input means for inputting a voice signal, and a language model or an acoustic model for performing voice recognition processing that is adapted to the user. A speech recognition processing means for performing a first speech recognition process on the speech signal using the language model or the acoustic model, and a vocabulary for extracting a vocabulary constituting a recognition processing result of the speech recognition processing means Extraction means and transmission means for transmitting information representing the vocabulary together with the voice signal to a server that performs second voice recognition processing on the voice signal using the vocabulary extracted by the vocabulary extraction means as a recognition dictionary And comprising.

  In the speech recognition method of the present invention, a language model or an acoustic model for performing speech recognition processing that is adapted to the user is stored in the storage means, and the speech input means inputs a speech signal. A voice input step, a voice recognition processing means for performing a first voice recognition process on the voice signal using the language model or the acoustic model, and a vocabulary extraction means for the voice recognition. A vocabulary extraction step for extracting a vocabulary constituting the recognition processing result of the processing means; and a transmission means for performing a second speech recognition process on the speech signal using the vocabulary extracted by the vocabulary extraction means as a recognition dictionary. A transmission step of transmitting information representing the vocabulary together with the audio signal to a server to perform.

  In the speech recognition program of the present invention, a language model or acoustic model for performing speech recognition processing and adapted to the user is stored in the storage means, and a speech input module for inputting a speech signal; A speech recognition processing module that performs a first speech recognition process on the speech signal using the language model or the acoustic model, and a vocabulary extraction module that extracts a vocabulary constituting a recognition processing result of the speech recognition processing module A transmission module that transmits information representing the vocabulary together with the audio signal to a server that performs second audio recognition processing on the audio signal using the vocabulary extracted by the vocabulary extraction module as a recognition dictionary; Is provided.

  According to the communication terminal, the speech recognition method, and the speech recognition program of the present invention, the vocabulary constituting the result of the first speech recognition processing by the speech recognition processing means of the communication terminal is converted into the second speech recognition process. Send to server to do. Since the speech recognition processing means of the communication terminal uses a language model or an acoustic model adapted to the user, speech recognition processing customized for the user is possible. Since the vocabulary constituting the result of the speech recognition process is transmitted to the server and used as a recognition dictionary, the server can perform a large vocabulary and highly accurate speech recognition process after expanding the recognition dictionary. Therefore, it is possible to realize a speech recognition process that achieves both large vocabulary and high-accuracy speech recognition processing and customization of each language model or acoustic model for each user while reducing unknown words in the second speech recognition processing. It becomes.

  In the present invention, the language model may include a user dictionary based on user data existing in the communication terminal or language data obtained from the user's usage history and depending on the user.

  According to this invention, the language model of the communication terminal can be a language model customized by the user. The user dictionary can include, for example, the name of a user acquaintance.

  In the present invention, the acoustic model may be applied to the user by using the user's past input voice or call voice, or the results of acoustic training.

  According to the present invention, a specific method for customizing an acoustic model of a communication terminal according to a user is provided.

  In the communication terminal of the present invention, the server may be connected to a network.

  According to this invention, since the communication terminal side transmits only the vocabulary constituting the result of the first speech recognition process to the server, not the entire user dictionary, the communication terminal and the server are connected on the network. In addition, there is an advantage that the cost of information transmission can be reduced. Further, since the amount of data to be transmitted is small, there is an advantage that the entire processing time is shortened and the delay time until the voice recognition processing is completed can be shortened.

  In the present invention, the vocabulary extracting means may extract only the vocabulary existing in the user data or the user dictionary.

  According to this invention, the vocabulary existing in the user data or user dictionary of the communication terminal can be reliably expanded in the recognition dictionary of the server. Further, the vocabulary to be extracted by the vocabulary extraction means can be reduced, and the amount of data to be transmitted from the communication terminal to the server can be further reduced.

  In the present invention, the transmitting means may further transmit information indicating whether or not the vocabulary exists in the user dictionary to the server.

  According to this invention, by referring to the information, the vocabulary existing in the user data or the user dictionary of the communication terminal can be reliably specified, and the vocabulary can be reliably expanded in the recognition dictionary of the server.

  According to the present invention, while reducing unknown words in the second speech recognition process, a speech recognition process that achieves both large vocabulary and high-accuracy speech recognition processing and customization of a language model or acoustic model for each user is realized. It is possible to provide a communication terminal, a speech recognition method, and a speech recognition program that can be used.

1 is a schematic configuration diagram of a voice recognition system 1. FIG. 2 is a hardware configuration diagram of a client terminal 100 and a server 200. FIG. It is a sequence diagram which shows the operation | movement performed with the speech recognition system. It is a figure which shows an example of the result of the 1st audio | voice recognition process by the client terminal.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of a communication terminal, a speech recognition method, and a speech recognition program according to the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. The “speech recognition process” in the following description is a process of analyzing a spoken language spoken by a speaker and extracting the content being spoken as character data.

(Overall configuration of the speech recognition system 1)
First, the configuration of the speech recognition system 1 according to the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a schematic configuration diagram of the speech recognition system 1. As shown in FIG. 1, the speech recognition system 1 includes a client terminal 100 (corresponding to a “communication terminal” in the claims) and a server 200, and the client terminal 100 and the server 200 are connected on a network 300. ing. In FIG. 1, only one client terminal 100 is shown as a representative, but a plurality of client terminals 100 can communicate with the server 200. The client terminal 100 performs a first voice recognition process, and the server 200 performs a second voice recognition process. The result of the second voice recognition process is the final result of the voice recognition process.

(Configuration of client terminal 100)
The client terminal 100 will be described in detail. The client terminal 100 is a mobile phone or a smartphone, for example, and FIG. 2 is a hardware configuration diagram of the client terminal 100. As shown in FIG. 2, the client terminal 100 physically includes a CPU 11, a ROM 12 and a RAM 13 which are main storage devices, an input device 14 such as operation buttons and a microphone, an output device 15 such as an LCD or an organic EL display, and a server. A communication module 16 that transmits and receives data to and from the memory 200 and an auxiliary storage device 17 such as a memory device are provided. Each function of the client terminal 100 to be described later operates the input device 14, the output device 15, and the communication module 16 under the control of the CPU 11 by reading predetermined software on hardware such as the CPU 11, the ROM 12, and the RAM 13. At the same time, it is realized by reading and writing data in the main storage devices 12 and 13 and the auxiliary storage device 17.

  Returning to FIG. 1, the client terminal 100 functionally includes a voice input unit 110 (corresponding to “voice input unit” in the claims) and a terminal-side storage unit 120 (“storage unit” in the claims). Equivalent), terminal-side voice recognition unit 130 (corresponding to “voice recognition processing means” in the claims), vocabulary extraction unit 140 (corresponding to “vocabulary extraction means” in the claims), and transmission unit 150 (patent) Equivalent to “transmission means” in the claims).

  The voice input unit 110 inputs a voice signal from a user, and can be configured by the input device 14 shown in FIG. 2, for example. The voice input unit 110 performs A / D conversion on a voice signal input from a microphone to generate voice data. The voice input unit 110 may further generate encoded data by converting the generated voice data into a compression code, or may extract feature data from the voice data. Hereinafter, the encoded data and the feature data are referred to as audio data. The voice input unit 110 outputs the generated voice data to the terminal-side voice recognition unit 130 and the transmission unit 150.

  The terminal-side storage unit 120 stores a language model or an acoustic model adapted for the user, which is a language model or an acoustic model for the terminal-side speech recognition unit 130 to perform speech recognition processing. In many cases, the client terminal 100 according to the present embodiment is dedicated to a user or a communication terminal that can be used only by a limited user. Therefore, in this case, personal customization of the language model and acoustic model is easy. The terminal-side language model stored in the terminal-side storage unit 120 includes a user dictionary based on user data existing in the client terminal 100 or language data obtained from the user's usage history and depending on the user. It can be customized by the user of the terminal 100. In addition, the terminal-side acoustic model stored in the terminal-side storage unit 120 may be customized by being adapted to the user using the user's past input voice or call voice, or the results of acoustic training. it can. Such a terminal-side storage unit 120 can be configured by the auxiliary storage device 17 shown in FIG. 2, for example.

  The terminal-side speech recognition unit 130 is configured by, for example, the CPU 11 shown in FIG. 2 and the like, and uses the language model or the acoustic model stored in the terminal-side storage unit 120 for voice data input from the speech input unit 110. The first voice recognition process is performed. Since the first speech recognition process is a speech recognition process on the client terminal 100 side, it is not a large vocabulary and high-accuracy speech recognition process that uses abundant amounts of memory and computing resources. This is a speech recognition process adapted to a user using a model or an acoustic model. Note that the speech recognition process itself performed by the terminal-side speech recognition unit 130 is a well-known technique, for example, using a unigram or bigram as a language model and a hidden Markov model as an acoustic model. Therefore, detailed description is omitted here. The voice recognition in the terminal side voice recognition unit 130 may be continuous voice recognition or isolated word recognition for the entire input voice, or may be voice recognition or word spotting for a part of the voice. The terminal-side voice recognition unit 130 outputs the result of the first voice recognition process to the vocabulary extraction unit 140.

  The vocabulary extraction unit 140 receives the result of the first speech recognition processing from the terminal side speech recognition unit 130 and extracts the vocabulary (notation and reading pair) constituting the result. The vocabulary extraction unit 140 can be configured by, for example, the CPU 11 shown in FIG. The vocabulary extraction unit 140 may extract all of the vocabulary constituting the result of the first speech recognition process, or may extract only the vocabulary existing in the user data or the user dictionary. Alternatively, when the extracted vocabulary is present in the user data or the user dictionary, the vocabulary extraction unit 140 generates information indicating the fact (hereinafter referred to as “instruction signal”) and the vocabulary extraction processing. May be performed. If there is an instruction signal, vocabulary extraction section 140 outputs information indicating the extracted vocabulary (hereinafter referred to as “vocabulary information”) to transmission section 150 together with the instruction signal.

  The transmission unit 150 transmits the instruction signal to the server 200 if there is vocabulary information input from the vocabulary extraction unit 140 and an instruction signal together with the voice data input from the voice input unit 110. The transmission unit 150 can be configured by, for example, the communication module 16 illustrated in FIG.

(Configuration of server 200)
Next, the server 200 will be described. FIG. 2 is a hardware configuration diagram of the server 200. As shown in FIG. 2, the server 200 is physically connected between a CPU 21, a main storage device such as a ROM 22 and a RAM 23, an input device 24 such as a keyboard and a mouse, an output device 25 such as a display, and a client terminal 100. The computer system includes a communication module 26 such as a network card for transmitting and receiving data, an auxiliary storage device 27 such as a hard disk, and the like. Each function of the server 200 described later causes the input device 24, the output device 25, and the communication module 26 to operate under the control of the CPU 21 by reading predetermined computer software on the hardware such as the CPU 21, the ROM 22, and the RAM 23. At the same time, it is realized by reading and writing data in the main storage devices 22 and 23 and the auxiliary storage device 27.

  Returning to FIG. 1, the server 200 is functionally configured to include a receiving unit 210, a server-side storage unit 220, a recognition dictionary expansion unit 230, a server-side speech recognition unit 240, and a recognition result transmission unit 250.

  The receiving unit 210 receives the instruction signal from the transmission unit 150 of the client terminal 100 if there is voice data, vocabulary information, and an instruction signal. The receiving unit 210 outputs the received voice data to the server-side voice recognition unit 240, and outputs the received vocabulary information and instruction signal to the recognition dictionary expansion unit 230.

  The server-side storage unit 220 stores a language model or an acoustic model for the server-side speech recognition unit 240 to perform speech recognition processing. Since the server 200 in this embodiment is often shared by many users, it is not easy to personally customize a language model or an acoustic model on the server side, but the server side storage unit 220 has a large vocabulary. It is also possible to store a language model or acoustic model suitable for high-accuracy speech recognition processing.

  The recognition dictionary expansion unit 230 receives the vocabulary information and the instruction signal from the reception unit 210 if there is an instruction signal. Based on the input information, the recognition dictionary expansion unit 230 receives the server-side language model stored in the server-side storage unit 220. It extends the recognition dictionary. “Expanding the server-side language model recognition dictionary” means that when the vocabulary represented by the vocabulary information input from the receiving unit 210 is an unknown word in the server-side language model, the unknown word is converted to the server-side language model. By registering as a known word in the recognition dictionary, this means that the vocabulary is no longer unknown. Such an expansion of the recognition dictionary can reduce unknown words when the server side speech recognition unit 240 performs speech recognition processing using the server side language model.

  The server-side voice recognition unit 240 performs a second voice recognition process on the voice data input from the reception unit 210 using the language model or the acoustic model stored in the server-side storage unit 220. Since the second speech recognition process is a speech recognition process on the server 200 side, a large vocabulary and high-accuracy speech recognition process using a large amount of memory and computing resources is possible. Further, the second speech recognition process is performed using the language model after the recognition dictionary is expanded by the recognition dictionary expansion unit 230. For this reason, it is possible to perform speech recognition processing after reducing unknown words, and the accuracy of the recognition result is improved. Note that the speech recognition processing itself by the server-side speech recognition unit 240 is a well-known technique, for example, using a trigram as a language model and a hidden Markov model as an acoustic model. Omitted. The server side voice recognition unit 240 outputs the result of the second voice recognition processing to the recognition result transmission unit 250.

  The recognition result transmission unit 250 receives the result of the second speech recognition processing from the server side speech recognition unit 240 and transmits the result to the client terminal 100. The client terminal 100 includes means for receiving the result of the second speech recognition process (for example, the communication module 16 shown in FIG. 2) and means for displaying the result to the user (for example, the output device 15 shown in FIG. 2). Possible).

(Operation of the speech recognition system 1)
Next, operations performed by the voice recognition system 1 will be described with reference to FIG. FIG. 3 is a sequence diagram showing operations performed in the speech recognition system 1.

(Step S1, equivalent to “voice input step” in the claims)
First, after the voice input unit 110 of the client terminal 100 receives a voice signal from the user and performs A / D conversion or the like, the voice data is output to the terminal-side voice recognition unit 130 and the transmission unit 150.

(Step S2, corresponding to “voice recognition processing step” in the claims)
Next, the terminal side voice recognition unit 130 uses the language model or the acoustic model stored in the terminal side storage unit 120 to perform first voice recognition on the voice data input from the voice input unit 110 in step S1. Process. FIG. 4 shows an example of the result of the first speech recognition process. Although the recognition result is indicated by N-best in FIG. 4, the recognition result is not limited to this, and may be a confusion network or a word lattice. The terminal side speech recognition unit 130 outputs the result of the first speech recognition process as shown in FIG. 4 to the vocabulary extraction unit 140.

(Step S3, equivalent to “vocabulary extraction step” in the claims)
Next, the vocabulary extraction unit 140 receives the result of the first speech recognition process in step S2 from the terminal-side speech recognition unit 130, and extracts the vocabulary constituting the result. When the recognition result as in the example of FIG. 4 is input, the vocabulary extraction unit 140 extracts the following vocabulary and outputs vocabulary information representing the extracted vocabulary to the transmission unit 150 (the vocabulary extraction unit 140 Vocabulary extraction pattern 1).
“Today / Kyo”, “Ha / Wa”, “Yokohama / Yokohama”, “Takahama / Takahama”, “Yokoshima / Yokoshima”, “He / E”, “Ni / Ni”, “Done / Itta”, “ Meet / Atta ”

  Here, the vocabulary extraction unit 140 may extract only the vocabulary in the recognition result shown in FIG. 4 that exists in the user data or the user dictionary. For example, in the example of FIG. 4, only “Yokoshima / Yokoshima” is a vocabulary existing in the user data or user dictionary of the client terminal 100, and other “today / kyo”, “ha / wa”, etc. are user data or It is assumed that the vocabulary does not exist in the user dictionary. In this case, the vocabulary extraction unit 140 extracts only “Yokoshima / Yokoshima” and outputs lexical information representing the extracted vocabulary “Yokoshima / Yokoshima” to the transmission unit 150 (vocabulary extraction pattern 2 of the vocabulary extraction unit 140). .

  Further, when the extracted vocabulary exists in the user data or the user dictionary, the vocabulary extraction unit 140 generates information (instruction signal) to that effect, as well as the vocabulary extraction pattern 1 described above. Vocabulary extraction processing may be performed. For example, in the example of FIG. 4, the vocabulary extraction unit 140 extracts all vocabulary in recognition results such as “today / kyo”, “ha / wa”, and the vocabulary “Yokoshima / Yokoshima” is user data or a user dictionary. An instruction signal indicating that it is a vocabulary existing in is generated. Then, the vocabulary extraction unit 140 outputs vocabulary information representing the extracted vocabulary together with the instruction signal to the transmission unit 150 (vocabulary extraction pattern 3 of the vocabulary extraction unit 140).

(Step S4, corresponding to “transmission step” in claims)
Next, if there is vocabulary information input from the vocabulary extraction unit 140 in step S3 and an instruction signal together with the audio data input from the audio input unit 110 in step S1, the transmission unit 150 sends the instruction signal to the server 200. Send.

(Step S5)
Next, the receiving unit 210 of the server 200 receives the instruction signal from the transmitting unit 150 of the client terminal 100 if there is voice data, vocabulary information, and an instruction signal. The receiving unit 210 outputs the received voice data to the server-side voice recognition unit 240, and outputs the received vocabulary information and instruction signal to the recognition dictionary expansion unit 230.

(Step S6)
Next, if there is vocabulary information and an instruction signal from the receiving unit 210, the recognition dictionary expansion unit 230 inputs the instruction signal, and the server side stored in the server side storage unit 220 based on the input information. Extend the language model recognition dictionary.

  When the vocabulary information that represents all the vocabulary extracted by the vocabulary extraction unit 140 is input as in the vocabulary extraction pattern 1 described above, the recognition dictionary expansion unit 230, the vocabulary represented by the input vocabulary information, Compare the vocabulary registered in the recognition dictionary held by the server and register only the vocabulary represented by the vocabulary information that is not registered in the server's recognition dictionary as known words. May be. In the example of FIG. 4, when it is determined that, for example, two vocabularies “Yokohama / Yokohama” and “Takahama / Takahama” are not registered in the server's recognition dictionary, the recognition dictionary expansion unit 230 The vocabulary “Yokohama / Yokohama” and “Takahama / Takahama” are newly registered as known words in the recognition dictionary of the own server (dictionary expansion pattern 1 of the recognition dictionary expansion unit 230).

  Alternatively, when it is ensured that the vocabulary represented by the input vocabulary information exists in the user data or user dictionary of the client terminal 100 as in the vocabulary extraction pattern 2 described above, the recognition dictionary The extension unit 230 may register all the vocabulary represented by the input vocabulary information in the recognition dictionary as known words. In such a case, for example, in the example of FIG. 4, the fact that the vocabulary extraction unit 140 of the client terminal 100 extracts only the vocabulary existing in the user data or the user dictionary is, for example, predetermined information between the client terminal 100 and the server 200. This is a case where the server 200 knows in advance by exchanging. In such a case, for example, only “Yokoshima / Yokoshima” is extracted by the vocabulary extraction unit 140, lexical information representing only “Yokoshima / Yokoshima” is input to the recognition dictionary expansion unit 230, and the recognition dictionary expansion unit 230 Register “Yokoshima / Yokoshima” as a known word in the recognition dictionary of the local server. Note that if the vocabulary “Yokoshima / Yokoshima” is a known word already registered in the recognition dictionary of the server language model, the recognition dictionary expansion unit 230 may not register the vocabulary “Yokoshima / Yokoshima”. (Dictionary expansion pattern 2 of the recognition dictionary expansion unit 230).

  Further, when there is an instruction signal as in the vocabulary extraction pattern 3 described above, the recognition dictionary expansion unit 230 refers to the instruction signal so that the vocabulary exists in the user data or the user dictionary of the client terminal 100. The vocabulary may be registered in the recognition dictionary as a known word only when it is found to be a thing. For example, in the example of FIG. 4 described above, when an instruction signal indicating that “Yokoshima / Yokoshima” is a vocabulary existing in the user data or the user dictionary is input, the recognition dictionary expansion unit 230 uses the vocabulary “Yokoshima / Yokoshima”. Is registered in the recognition dictionary of its own server as a known word. Note that if the vocabulary “Yokoshima / Yokoshima” is a known word already registered in the recognition dictionary of the server language model, the recognition dictionary expansion unit 230 may not register the vocabulary “Yokoshima / Yokoshima”. (Dictionary expansion pattern 3 of the recognition dictionary expansion unit 230).

(Step S7)
Next, the server side voice recognition unit 240 uses the language model or the acoustic model stored in the server side storage unit 220 to perform the second voice recognition process on the voice data input from the reception unit 210 in step S5. I do. The second speech recognition process is performed using the language model after the recognition dictionary is expanded by the recognition dictionary expansion unit 230. The server side voice recognition unit 240 outputs the result of the second voice recognition processing to the recognition result transmission unit 250.

(Step S8)
Next, the recognition result transmission unit 250 receives the result of the second speech recognition processing from the server side speech recognition unit 240 and transmits the result to the client terminal 100.

(Step S9)
Next, the client terminal 100 side receives the result of the second voice recognition process and displays the result to the user.

  In the above description, the speech recognition system 1 including the client terminal 100 and the server 200 is illustrated as an embodiment of the present invention. However, the present invention is not limited to this, and each function in the client terminal 100 and the server 200 is executed. You may comprise as a program provided with the module for. That is, as the equivalent to the client terminal 100, a language model or an acoustic model for performing speech recognition processing that is adapted to the user is stored in the storage means, and the speech corresponding to the speech input unit 110 is stored. A program including an input module, a speech recognition processing module corresponding to the terminal-side speech recognition unit 130, a vocabulary extraction module corresponding to the vocabulary extraction unit 140, and a transmission module corresponding to the transmission unit 150 is configured. Similarly, a program including each module corresponding to each component of the server 200 is configured. And a function equivalent to the voice recognition system 1 provided with the above-mentioned client terminal 100 and the server 200 is realizable by making a portable terminal, a smart phone, a server, etc. read the said program. Such a program can be recorded on a recording medium. A recording medium is a signal format that causes a state of change in energy such as magnetism, light, electricity, etc. according to the description of a program to a reading device provided in a computer hardware resource. Thus, the description content of the program can be transmitted to the reading device. Examples of such a recording medium include a magnetic disk, an optical disk, a CD-ROM, and a memory built in a computer.

(Operation and effect of this embodiment)
Then, the effect | action and effect of the speech recognition system 1 concerning this embodiment are demonstrated. According to the speech recognition system 1 of the present embodiment, the vocabulary constituting the result of the first speech recognition process by the terminal-side speech recognition unit 130 of the client terminal 100 is transmitted to the server 200 that performs the second speech recognition process. . Since the terminal-side voice recognition unit 130 of the client terminal 100 uses a language model or an acoustic model adapted to the user, voice recognition processing customized for the user is possible. Since the vocabulary constituting the result of the speech recognition process is transmitted to the server 200 and used as a recognition dictionary, the server 200 can perform a large vocabulary and highly accurate speech recognition process after expanding the recognition dictionary. Become. Therefore, it is possible to realize a speech recognition process that achieves both large vocabulary and high-accuracy speech recognition processing and customization of each language model or acoustic model for each user while reducing unknown words in the second speech recognition processing. It becomes.

  According to the present embodiment, the language model of the client terminal 100 includes a user dictionary based on user data existing in the client terminal 100 or language data obtained from the user usage history and depending on the user. Thus, the language model of the client terminal 100 can be a language model customized by the user. The user dictionary can include, for example, the name of a user acquaintance.

  In addition, according to the present embodiment, a specific method for customizing the acoustic model of the client terminal 100 according to the user, such as using the user's past input voice or call voice, or the results of acoustic training, is provided. Is done.

  Further, according to the present embodiment, the client terminal 100 transmits only the vocabulary constituting the result of the first speech recognition process to the server 200 instead of the entire user dictionary, so that the client terminal 100 and the server 200 are connected to the network. In the case of this embodiment connected above, there is an advantage that the cost of information transmission can be reduced. Further, since the amount of data to be transmitted is small, there is an advantage that the entire processing time is shortened and the delay time until the voice recognition processing is completed can be shortened.

  Further, according to the present embodiment, particularly in the vocabulary extraction pattern 2 and the dictionary expansion pattern 2, the vocabulary existing in the user data of the client terminal 100 or the user dictionary can be reliably expanded in the recognition dictionary of the server 200. . Further, the vocabulary to be extracted by the vocabulary extraction unit 140 can be reduced, and the amount of data to be transmitted from the client terminal 100 to the server 200 can be further reduced.

  Further, according to the present embodiment, particularly in the vocabulary extraction pattern 3 and the dictionary expansion pattern 3, by referring to the instruction signal, the vocabulary existing in the user data or the user dictionary of the client terminal 100 can be reliably identified, The vocabulary can be reliably expanded in the recognition dictionary of the server 200.

DESCRIPTION OF SYMBOLS 1 ... Voice recognition system, 100 ... Client terminal, 110 ... Voice input part, 120 ... Terminal side storage part, 130 ... Terminal side voice recognition part, 140 ... Vocabulary extraction part, 150 ... Transmission part, 200 ... Server, 210 ... Reception , 220 ... server-side storage unit, 230 ... recognition dictionary expansion unit, 240 ... server-side voice recognition unit, 250 ... recognition result transmission unit, 300 ... network.

Claims (7)

A speech recognition system that performs speech recognition processing by communication between a communication terminal and a server,
The communication terminal is
A voice input means for inputting a voice signal;
Storage means for storing a language model or an acoustic model adapted to the user for performing speech recognition processing;
Speech recognition processing means for performing a first speech recognition process on the speech signal using the language model or the acoustic model;
Vocabulary extracting means for extracting a vocabulary constituting a recognition processing result of the speech recognition processing means and generating an instruction signal which is information indicating whether or not the extracted vocabulary is present in the language model ;
Before SL lexical information is information representing the vocabulary along with audio signals, and said indication signal and a transmitting means for transmitting to said server,
The server
A server-side storage for storing a server-side language model, which is a language model for performing speech recognition processing, which is a language model adapted to speech recognition processing with a large vocabulary and higher accuracy than the language model stored in the storage means Means,
Recognition dictionary expansion means for registering an unknown word in the server-side language model as a known word based on the vocabulary information and the instruction signal;
Server-side speech recognition processing means for performing a second speech recognition process on the speech signal using the server-side language model;
A speech recognition system comprising: The language model stored by the storage unit includes user data existing in the communication terminal or a user dictionary based on language data obtained from the user's usage history and depending on the user.
The speech recognition system according to claim 1. The acoustic model stored by the storage unit is adapted to the user by using the past input voice or call voice of the user, or the results of acoustic training,
The voice recognition system according to claim 1, wherein The server is connected to a network,
The speech recognition system according to any one of claims 1 to 3, wherein The vocabulary extracting means extracts only the vocabulary existing in the user data or the user dictionary;
Speech recognition system according to claim 2, characterized in that. A speech recognition method for performing speech recognition processing by communication between a communication terminal and a server,
In the storage means of the communication terminal, a language model or an acoustic model for performing speech recognition processing that is adapted to the user is stored,
A voice input step in which the voice input means of the communication terminal inputs a voice signal;
A voice recognition processing step in which voice recognition processing means of the communication terminal performs a first voice recognition process on the voice signal using the language model or the acoustic model;
The vocabulary extracting means of the communication terminal extracts the vocabulary constituting the recognition processing result of the speech recognition processing means, and an instruction that is information indicating whether or not the extracted vocabulary exists in the language model A vocabulary extraction step for generating a signal ;
Transmitting means of said communication terminal, before Symbol vocabulary information which is information representing the vocabulary along with audio signals, and and a transmission step of transmitting the instruction signal to the server,
The server side which is a language model for performing speech recognition processing in the server storage means of the server, and which is a language model suitable for speech recognition processing having a larger vocabulary and higher accuracy than the language model stored in the storage means Language model is stored,
A recognition dictionary expansion means for registering an unknown word in the server side language model as a known word based on the vocabulary information and the instruction signal;
A server side speech recognition processing step in which the server side speech recognition processing means of the server performs a second speech recognition process on the speech signal using the server side language model;
A speech recognition method comprising: A speech recognition program for speech recognition processing performed by communication between a communication terminal and a server,
An audio input module for inputting audio signals;
A speech recognition processing module that performs a first speech recognition process on the speech signal using a language model or an acoustic model that is stored in the communication terminal and that is adapted to the user. When,
A vocabulary extraction module that extracts a vocabulary constituting a recognition processing result of the speech recognition processing module and generates an instruction signal that is information indicating whether the extracted vocabulary is present in the language model ;
A transmitting module for sending pre SL lexical information is information representing the vocabulary along with audio signals, and said indication signal to said server,
A language model for performing speech recognition processing based on the vocabulary information and the instruction signal, and a language model adapted to speech recognition processing having a larger vocabulary and higher accuracy than a language model stored in the communication terminal. A recognition dictionary extension module for registering unknown words in the server-side language model as known words;
A server-side speech recognition processing module that performs a second speech recognition process on the speech signal using the server-side language model;
A speech recognition program comprising:

Images