JP6277659B2 - Speech recognition apparatus and speech recognition method - Google Patents

Description

  The present invention relates to a speech recognition technique for improving speech recognition accuracy by improving a word chain coverage by adapting a language model to input speech.

Speech recognition technology that makes speech into text is useful, and improves text input efficiency and facilitates database search by text input in many fields, such as transcription of spoken speech in medical and legal fields and creation of subtitles in broadcasting etc. Application to computerization is expected or already applied.
However, there is a possibility that misrecognition is included in the speech recognition result, and how to reduce misrecognition is a big problem in speech recognition technology. In current speech recognition technology, an acoustic model that associates speech features with phonemes and a language model that expresses the relationship between linked words are generally used.

  The technical terms used in the following are: Prior Art Document 1: Kiyohiro Shikano, Katsunobu Ito, Tatsuya Kawahara, Kazuya Takeda, Mikio Yamamoto, “Voice Recognition System”, Ohm Corporation, May 15, 2001, p. .53-175 (hereinafter referred to as textbook 1), or prior art reference 2: Kenji Kita, Junichi Sakurai, “Probabilistic Language Model”, University of Tokyo Press, November 25, 1999, p.57-99 (hereinafter referred to as textbook) 2) or prior art document 3: Nagao Makoto, “Natural Language Processing”, Iwanami Shoten, April 26, 1996, p.118-137 (hereinafter referred to as textbook 3).

  A method using an N-gram model described in textbooks 1 to 3 as a language model for accurately recognizing speech has attracted attention. In the language model of the N-gram model, since N-grams are learned from the corpus, it is known that there is a sparseness problem that word chains that do not appear in the corpus cause misrecognition.

  In order to cope with this sparseness problem, a technique for adapting a language model to input speech has been proposed. For example, in Patent Document 1, based on the first speech recognition result, an appropriate language model is selected from previously layered language models, and the selected language models are mixed to generate one language model. A technique for adapting a language model to input speech is disclosed.

WO2008 / 004666 (Fig. 3)

  In the above-described conventional speech recognition apparatus, word chains that have not been learned in any of the selected language models included in the input speech are unlearned word chains in the mixed language model, and those unlearned words There was a problem that the possibility of misrecognizing a word sequence is still high.

  The present invention has been made to solve the above-described problems, and it is possible to improve the coverage of word chains that were not included in the original language model even after learning word chains of speech recognition results. An object of the present invention is to obtain a speech recognition device having a simple language model.

Speech recognition apparatus of this invention, a second language model storage unit for storing a mixed language model, the mixing using a language model to recognize the input speech signal a second outputting a second speech recognition result a voice recognition unit, by using the first language model to recognize the input speech signal, a first speech recognition unit which outputs the first speech recognition result, the first pre-SL by the first speech recognition section speech recognition result is input, to create a N-gram based on the word sequence of the first speech recognition result, the first result speech recognition stored in said second language model storage unit the mixed language model And an N-gram adding unit for adding N-grams created based on the word series .

According to the speech recognition method of the present invention, the first speech recognition unit performs speech recognition of speech input by referring to the first language model, and the second speech recognition unit refers to the mixed language model. A second speech recognition procedure for performing speech recognition of the input speech, and an N-gram adding unit creates an N-gram based on a word sequence of a speech recognition result by the first speech recognition procedure, and the mixed language And an N-gram adding procedure for adding an N-gram created based on the word sequence of the first speech recognition result to the model.

As described above, according to the speech recognition apparatus of the present invention, the second language model is input by updating the second language model that is the mixed language model based on the word sequence of the recognition result of the first language model. It is possible to improve the speech recognition performance by improving the coverage of word chains appearing in the input speech by adapting to the speech.
According to the speech recognition method according to the present invention, by performing the procedure of updating the second language model that is a mixed language model based on the word sequence of the speech recognition result input by the first speech recognition procedure, By applying the second language model to the input speech, the coverage of word chains appearing in the input speech can be improved, and speech recognition performance can be improved.

It is a block diagram which shows the structure of the speech recognition apparatus of Embodiment 1 of this invention. 3 is a flowchart of processing performed by the speech recognition apparatus according to the first embodiment. It is a figure which shows the example of the word series of the recognition result of the 1st speech recognition part of the speech recognition apparatus of Embodiment 1. FIG. It is a figure which shows the example of the learning example sentence information before the update of the 2nd speech recognition part of the speech recognition apparatus of Embodiment 1. FIG. It is a figure which shows the example of N-gram before the update of the 2nd language model of the speech recognition apparatus of Embodiment 1. FIG. It is a figure which shows the example of the learning example sentence information after the update of the 2nd speech recognition part of the speech recognition apparatus of Embodiment 1. FIG. It is a figure which shows the example of the N-gram after the 2nd language model of the speech recognition apparatus of Embodiment 1 is updated. It is a figure which shows the example of the probability of each word of the correct word series based on the 2nd language model after the update of the operation example of the speech recognition apparatus of Embodiment 1. FIG. It is a figure which shows the example of the probability of each word of the word series containing the error based on the 2nd language model after the update of the operation example of the speech recognition apparatus of Embodiment 1. It is a figure which shows the comparison of the probability of the word of the language model which mixed the 2nd language model after the update of the character string search apparatus of Embodiment 1, a 1st language model, and the 2nd language model before an update. FIG. 10 is a block diagram showing a configuration of a modified example in which the second language model is updated with the recognition result of the second speech recognition unit in the first embodiment. FIG. 10 is a diagram illustrating an example of a learning example sentence when the second language model is updated with the recognition result of the second speech recognition unit in the first embodiment. 6 is a diagram illustrating an example of a second language model updated with a recognition result of a second speech recognition unit in Embodiment 1. FIG. FIG. 10 is a diagram showing a comparison of word probabilities of the second language model when updated with the recognition result of the first speech recognition unit and updated with the recognition result of the second speech recognition unit in the first embodiment. It is a block diagram which shows the structure of the speech recognition apparatus of Embodiment 2 of this invention. 6 is a flowchart of processing performed by the speech recognition apparatus according to the second embodiment. It is a figure which shows the example of the recognition result of the 3rd speech recognition part of the speech recognition apparatus of Embodiment 2. FIG. It is a figure which shows the example of the learning example sentence which the N-gram addition part of the speech recognition apparatus of Embodiment 2 updated. It is a figure which shows the example of the 2nd language model updated of the speech recognition apparatus of Embodiment 2. FIG. It is a figure which shows the example of the 2nd language model updated of the speech recognition apparatus of Embodiment 2. FIG. It is a block diagram which shows the structure of the speech recognition apparatus of Embodiment 3 of this invention. It is a figure which shows the example of the learning example sentence information which weighted the speech recognition apparatus of Embodiment 3. FIG. It is a figure which shows the example of the learning example sentence information which weighted the speech recognition apparatus of Embodiment 3. FIG.

  Embodiments of the present invention will be described below with reference to the drawings. In the drawings to be referred to, the same or corresponding parts are denoted by the same reference numerals.

The specific example of the operation shown below is an example of a speech recognition engine that performs speech recognition processing. Julius-4.2.2 (http://julius.sourceforge.jp, which is an open source large vocabulary continuous speech recognition engine) Then, we will explain using experimental results using Julius-4.2.2.
For example, the acoustic model is hmmdefs_ptm_gid.binhmm, which is included in the Julius dictation execution kit, and the NMU probability in the language model update process (hereinafter referred to as language model creation tool) is a CMU described in textbook 1, for example. -Cambridge statistical language model toolkit can be used.
In the following embodiment, the case where the order of N-gram is 3 (N = 3) will be described. However, the present invention does not limit the order of N-grams to 3, but may be 2 grams or N-grams having multiple orders of 4 or more.

Embodiment 1 FIG.
1 is a diagram showing a configuration of a speech recognition apparatus according to Embodiment 1 of the present invention. The speech recognition apparatus according to Embodiment 1 includes a speech input unit 101, a first speech recognition unit 102, a first language model storage unit 103, a first acoustic model storage unit 104, an N-gram addition unit 105, and a second speech recognition unit 106. , A second language model (mixed language model) storage unit 107 and a second acoustic model storage unit 108.

The voice input unit 101 converts voice uttered by the user into a digital voice signal that can be digitally processed. The digital voice signal output from the voice input unit 101 is input to the first voice recognition unit 102 and the second voice recognition unit 106.
The first speech recognition unit 102 refers to the input digital speech signal with reference to the first acoustic model stored in the first acoustic model storage unit 104 and the first language model stored in the first language model storage unit 103. , Perform voice recognition. The first speech recognition unit 102 outputs the word series (recognition sentence) of the speech recognition result to the N-gram adding unit 105. The N-gram adding unit 105 extracts the N-gram from the input word sequence of the speech recognition result, calculates the N-gram probability, and adds the N-gram to the second language model stored in the second language model storage unit 107. Add

  The second speech recognition unit 106 to which the digital speech signal is input from the speech input unit 101 is stored in the second language model that is a mixed language model stored in the second language model storage unit 107 and the second acoustic model storage unit 108. Speech recognition is performed with reference to the second acoustic model, and a word sequence of the speech recognition result is output. The word sequence of the speech recognition result output from the second speech recognition unit 106 is used in a process such as being displayed on the display unit 109, for example.

  In the above description, the acoustic model (first acoustic model, second acoustic model) is a database that holds patterns of standard feature amounts of speech in units of phonemes and the like. Examples of the voice feature amount include MFCC (Mel Frequency Cepstrum Coefficient) and ΔMFCC. The acoustic model is collated with the feature amount of the input speech in the speech recognition process, and is used for the recognition of the input speech on a phoneme basis. Here, both the first acoustic model and the second acoustic model use the same acoustic model described above as an example, but the first acoustic model and the second acoustic model may be different acoustic models.

  The language model (first language model, second language model) is a database that defines restrictions on word chaining (word chain occurrence probability), and is configured based on an N-gram model. Language models are referred to based on phoneme level recognition results using acoustic models in the speech recognition process.

  A known method may be applied to the process of obtaining the recognition sentence from the input speech by referring to the acoustic model and the language model performed by the speech recognition unit (the first speech recognition unit 102 and the second speech recognition unit 106). Here, as an example, Julius is used as described above.

  The voice input unit 101 in FIG. 1 can be configured by a circuit such as a microphone that records voice and an AD (Analog Digital) converter that converts analog voice into digital data. The first speech recognition unit 102, the N-gram addition unit 105, and the second speech recognition unit 106 can be configured by a processor and peripheral circuits such as a RAM (Random Access Memory) and software executed by the processor. is there. The first language model storage unit 103, the first acoustic model storage unit 104, the second language model storage unit 107, and the second acoustic model storage unit 108 can be configured by a storage device such as a hard disk.

  Alternatively, by adopting a client-server format, for example, the first speech recognition unit 102, the first language model storage unit 103, and the first acoustic model storage unit 104 are realized as server functions accessible via a network. The speech recognition unit 106, the second language model storage unit 107, the second acoustic model storage unit 108, and the N-gram addition unit 105 may be configured as client functions.

Here, the likelihood calculation in the speech recognition processing performed by the speech recognition units (the first speech recognition unit 102 and the second speech recognition unit 106) will be described. Now, it is assumed that the word sequence W is composed of n words w ₁ to w _n as defined by the following Equation 1. At this time, the probability (likelihood) of the word sequence W is expressed by Equation 2 below. Here, P (w _i | w _i−2 , w _i−1 ) is the probability that the word w _i appears after the word w _{i −2} and the word w _i−1 (the N-gram probability in the third-order N-gram). ).

The probability of the word sequence is expressed by Equation 2 above, but the determination of the word sequence with the highest likelihood is sufficient if the magnitude relationships of the probabilities of the candidate word sequences can be compared. Therefore, in actual processing, as shown in Equation 3 below, the probability (P (w _i | w _i−2 , w _i−1 )) of each word w _i included in the word sequence is expressed as a logarithm (log probability). In other words, the sum X (W) of logarithmic probabilities of each word series is obtained, and the process of comparing X (W) of each candidate word series is performed.

  A language model gives the probability of each word described above. Hereinafter, the likelihood based on the language model is referred to as language likelihood.

  In the speech recognition processing, the above-mentioned language likelihood and the likelihood based on the acoustic model (acoustic likelihood) are evaluated according to the total likelihood f (h) added according to the following Equation 4, and the maximum likelihood candidate is determined as the speech recognition result. And In Equation 4, h is the word sequence of the candidate speech recognition result, AC (h) is the acoustic likelihood for the word sequence h, LM (h) is the language likelihood for the word sequence h, and n is the number of words in the candidate word sequence. is there. LM_WEIGHT is a language model weight, and LM_PENALTY is a word delivery penalty. In the following description of the specific operation examples, the results when 8.0 and -2.0 are set are used.

Next, the operation of the speech recognition apparatus according to this embodiment will be described. FIG. 2 is a diagram illustrating a processing flow of the speech recognition apparatus according to the first embodiment. In addition, in the following operation explanation, a specific example will be explained with an example where the sentence “Shu-zu-kara-ko-do-goshi-gai” (notation is “shoe case color code please”) is spoken. To do.
First, the voice input unit 101 converts the voice uttered by the user into an analog electric signal (ST201), and the input voice, which is an analog electric signal, is converted into digital information by an AD converter (ST202).

  Next, the first speech recognition unit 102 performs speech recognition based on the digital information of the input speech (ST203). In the process of ST203, first, the feature amount of the voice is extracted from the digital information of the input voice at an appropriate time interval (for example, every 60 milliseconds). Then, using the extracted feature amount, the first acoustic model stored in the first acoustic model storage unit 104 is referred to, the phoneme level is recognized, and the recognition result candidate word sequence and the acoustic model of the word sequence are used. Obtain acoustic likelihood. Furthermore, the language likelihood based on the language model is obtained by referring to the first language model stored in the first language model storage unit 103 based on the word sequence of the recognition result candidate, and the calculation shown in Equation 4 is performed as described above. Based on the result, the acoustic likelihood and the language likelihood are comprehensively determined, and the word sequence having the highest degree of matching with the input speech (that is, the maximum likelihood) is obtained as a recognized sentence (first speech recognition result).

  When Julius recognizes the above-mentioned voice input “Shu-zu-kara-ko-do-goshi-gai”, the word sequence shown in FIG. 3 is output as a recognition result. As an example of the first language model, the NP12y.60k.4.arpa (60K word N-gram, learning data: Mainichi Shimbun newspaper article data " CD-Mainichi Shimbun 91-2002 edition ”, morphological analysis: chasen 2.2.1 + ipadic 2.4.1, text size: 350 million morphemes, vocabulary size: 60156). “Color code” is not learned in this language model and is an unknown word in this language model.

  In FIG. 3, <s> and </ s> are symbols indicating the beginning and end of a sentence, respectively. Also, for example, “Shi” in “Shi: Shi: Su: 227” is the recognized word notation, “Shi” is the reading, “S” is the original form, “227” is the part of speech code (part of speech code) Respectively. That is, the word sequence of the recognition result is “Please act from the shoe case”. In this recognition result, a recognition error has occurred in “color code” as “from action” (from: kara: to: 63 action: code: action: 505). This is because in the first language model, “color code” is an unknown word, and there are similar reading conflict candidates, and this conflict candidate has been determined to be the maximum likelihood.

  Next, the N-gram adding unit 105 adds an N-gram based on the word sequence of the recognition result of the first speech recognition unit 102 to the second language model stored in the second language model storage unit 107. Here, unlike the first language model, the second language model is a language model obtained by learning N-grams from specialized example sentences in a specific field. Further, it is assumed that the second language model storage unit 107 stores example sentences (learning example sentences) used for learning as learning example sentence information (information about example sentences used for learning N-grams). The learning example sentence information may be stored in a storage medium (learning example sentence information storage unit) such as a RAM (Random Access Memory) different from the second language model storage unit 107. In this embodiment, the example sentence itself is used as learning example sentence information. However, other information that can be used for calculating N-gram probabilities, such as information on the number of appearances of each word appearing in an example sentence, is used as learning example sentence information. It is also possible to do.

  Now, it is assumed that N-grams are learned from two example sentences “shoe case” and “color code” in the second language model. FIG. 4 shows these two example sentences stored as learning example sentence information. As in FIG. 3, <s> indicates the beginning of a sentence, and </ s> indicates the end of the sentence, and indicates notation, reading, original form, and part of speech code. FIG. 5 shows N-grams related to these two example sentences learned by the second language model. Now, since the order of N-gram is N = 3, 1 gram, 2 gram, and 3 gram are learned.

In FIG. 5, for example, “-0.9031 Color code: Color code: Color code: 507 0.0000” on line 4 of \ 1-grams is N-gram (“Color code”). The same notation, reading, original form, and part-of-speech code as in FIG. 3), and “-0.9031” is the logarithmic probability of this N-gram. “0.0000” means that when a higher-order N-gram does not exist in the language model, the probability of the lower-order N-gram does not exist using back-off smoothing based on the Good Turing estimation method. It is a logarithmic value of a back-off coefficient used in the process of estimating the probability.
For example, "-0.3010 <s> Color code: Color code: Color code: 507 </ s>" on the second line of \ 3-grams is "<s> Color code: Color code: Color code: 507 < / s> ”is N-gram (3 grams of“ sentence, color code, end of sentence ”), and“ −0.3010 ”is the N-gram probability. Note that in the N-gram language model of degree 3, there is no backoff coefficient because the probability of higher-order N-grams is not estimated using 3 grams.

Here, a process for estimating a higher-order N-gram probability from a lower-order N-gram probability using a back-off coefficient will be described. The calculation for estimating the third-order N-gram probability using the second-order N-gram probability is defined by the following pseudo program code (pseudo code).
P (wd3 | wd1, wd2) = if (trigram exists) p_3 (wd1, wd2, wd3)
else if (bigram wd1, wd2 exists) bo_wt_2 (wd1, wd2) * P (wd3 | wd2)
else P (wd3 | wd2)
In this pseudo code, wd1, wd2, and wd3 indicate words. As described above, P (wd3 | wd1, wd2) is a probability (probability of the word wd3) that wd3 is generated after wd1 and wd2. The same applies to P (wd3 | wd2). Further, p_3 (wd1, wd2, wd3) is a 3-gram probability of the word strings wd1, wd2, wd3. bo_wt_2 (wd1, wd2) is a 2-gram back-off coefficient of the word string wd1, wd2.
That is, if 3 grams of the word strings wd1, wd2, and wd3 exist in the language model, the probability of the 3 grams is the probability of the word wd3. If 3 grams of word strings wd1, wd2, and wd3 do not exist, but 2 grams of word strings wd1 and wd2 exist, the product of backoff coefficient of 2 grams of word strings wd1 and wd2 and P (wd3 | wd2) Is the probability of the word wd3. If there are no two grams of the word strings wd1 and wd2, P (wd3 | wd2) is the probability of the word wd3.

Similarly, the calculation for estimating the second order N-gram probability using the first order N-gram probability is defined by the pseudo code shown below.
P (wd2 | wd1) = if (bigram exists) p_2 (wd1, wd2)
else bo_wt_1 (wd1) * p_1 (wd2)
Here, p_2 (wd1, wd2) is a 2-gram probability of the word string wd1, wd2, and p_1 (wd2) is a 1-gram probability of the word string wd2. Bo_wt_1 (wd1) is a 1-gram back-off coefficient of the word string wd1.
In the above pseudo code, the product is defined so as to obtain a product, but when calculating with logarithmic probability, a sum is calculated instead of a product.

  In the process of adding the word sequence of the recognition result of the first speech recognition unit 102 to the second language model, first, the N-gram addition unit 105 adds the word sequence of the recognition result of the first speech recognition unit 102 to the learning example sentence, and then learns the example sentence. Is updated (ST204). As described above, when the first voice recognition unit 102 recognizes “Please act from the shoe case” for the voice input “I want to do this”, it is shown in FIG. The word sequence of the recognition result is added to the learning example sentence. FIG. 6 shows learning example sentence information to which “the action is requested from the shoe case”, which is a word series of the recognition result of the first speech recognition unit 102, is added. <S> Shoes: Shoes: Shoes: 507 Case: Case: Case: 507 From: Kara: From: 63 Action: Code: Action: 505 Request: Onegai: Request: 505 Shi: Shi: To: 227 Mas: Mass: Mas: 146 </ s> "has been added.

  Next, the N-gram adding unit 105 calculates an N-gram probability based on the updated learning example sentence (ST205), and updates the second language model stored in the second language model storage unit 107 with the calculated N-gram probability. (ST206). FIG. 7 shows the updated second language model. The N-gram probability was calculated using the language model tool kit described above. In FIG. 7, for example, in the case of \ 3-grams, the "-0.4771 </ s> <s> color code: color code: color code: 507" on the first line indicates that the N-gram probability is the value before the update shown in FIG. The version has been updated from -0.3010 to -0.4771. Also, in the second language model before the update shown in Fig. 5 such as "-0.3010 Request: Onegai: Request: 505 Shi: Shi: Yes: 227 Masashi: Mas: Mas: 146" on the 6th line N-grams are newly added based on the word sequence of the recognition result of the first speech recognition unit 102.

  Next, the second voice recognition unit 106 performs voice recognition (ST207). In the process of ST207, the second acoustic model stored in the second acoustic model storage unit 108 is extracted from the digital information of the input speech in the same manner as the first speech recognition unit 102 and based on the extracted feature amount. Then, referring to the second language model (mixed language model) stored in the second language model storage unit 107, the most likely word sequence is obtained as a recognized sentence (second speech recognition result).

Here, a specific example of likelihood calculation in the speech recognition processing performed by the second speech recognition unit 106 will be described with reference to FIG. In FIG. 8, only the word notation is shown for easy viewing of the table, and the reading and original form are expressed in a omitted form. The same applies to FIG. 9, FIG. 10, and FIG.
As the probability of the word used in the likelihood calculation, the N-gram value having the highest degree is adopted. For example, in the case of P (<s> |), which is the probability of the beginning of a sentence, the degree is 1 gram because there is no preceding word, and the value -0.6368 of "<s>" in \ 1-grams in Fig. 7 is adopted. . In the case of P (shoe case | <s>), which is the probability of the shoe case following the beginning of the sentence, it is 2 grams, so the value of "<s> shoe case: shoe case: shoe case: 507" in \ 2-grams -0.5441 is adopted as the log probability.

The next P (color code | <s>, shoe case) is 3 grams, but there is nothing corresponding to \ 3-grams, so the N-gram probability is estimated by the back smoothing described above. \ 2-grams has "<s> Shoe case: Shoe case: Shoe case: 507", so use this 2 gram back-off factor. However, since there are not 2 grams of \ Shoecase: Shoecase: Shoecase: 507 Color code: Color code: Color code: 507 in \ 2-grams, the probability of this 2-gram is also estimated by back smoothing. .
The specific estimated value is calculated by bo_wt_2 (<s>, Shoe case: Shoe case: Shoe case: 507) + bo_wt_1 (Shoe case: Shoe case: Shoe case: 507) + p_1 (Color code: Color code: Color code: 507) = (0.1761) + (-0.4046) + (-1.2109) =-1.4394. As described above, the estimated value -1.4394 is adopted as the logarithmic probability. Similarly, logarithmic probabilities of all words are acquired, and the logarithmic probabilities of each word are used to calculate the correct word sequence "Please give me the <s> shoe case color code </ s>" according to Equation 3 above. When the likelihood based on the language model is calculated, as shown in the table of FIG. 8, (−0.6368) + (− 0.5441) + (− 1.4394) + (− 1.6155) + (− 0.1761) + (− 0.301) + (− 0.301) =-5.0139.

  The acoustic likelihood of the correct word sequence in the acoustic model used this time is -9118.412109, and the overall likelihood of the correct word sequence "<s> Shoecase color code please </ s>" based on Equation 4 above The degree can be calculated as -9118.412109 + (-5.0139 * 8.0) + (-2.0 * 7) =-9172.52.

The word sequence output as the recognition result of the first speech recognition unit 102 is mixed with the second language model while including erroneous recognition, and the word sequence including the erroneously recognized word is determined to be the maximum likelihood. Recognition performance cannot be improved.
However, the acoustic likelihood of the error word sequence “<s> Please take action from the shoe case </ s>” is -9133.199219, and the language likelihood is (-0.6368) + (-0.5441) + (-0.301) + (-0.301) + (-0.301) + (-0.301) + (-0.301) + (-0.301) + (-0.301) =-3.2879, so the total likelihood is −9133.199219 + (− 3.2879 × 8.0) + (− 2.0 × 9) = − 9177.5. Therefore, since the overall likelihood of the correct word sequence is higher than the overall likelihood of the word sequence including an error, the word sequence including the error is rejected, and the correct word sequence can be obtained as a speech recognition result. .

  Next, a language model obtained by simply mixing the first language model and the second language model before update (referred to as a simple mixed model), and the second language model after update of this embodiment described above A comparison of is shown. In FIG. 10, each word based on the simple mixed language model for the correct word system example “<s> shoe case color code please </ s>” and the updated second language model of this embodiment is shown. The probability, the order of the N-gram, and the language likelihood are shown. In the simple mixed language model, X (W) =-11.91879, whereas in the updated second language model, X (W) =-5.0139. Therefore, the updated second language model is related to the same word sequence. Language likelihood is high. This is an effect obtained by updating the second language model by adding the word sequence of the recognition result of the first speech recognition unit 102 to the learning example sentence. The use of the updated second language model having a high language likelihood in the speech recognition process is less likely to cause a recognition error, and the recognition performance can be improved.

  In this embodiment, the recognition result of the first speech recognition unit 102 is additionally mixed with the second language model. However, the N model addition unit 105c is based on the recognition result of the second speech recognition unit 106 as shown in FIG. The learning example sentence may be updated, and an N-gram may be added to the second language model based on the updated learning example sentence. At this time, if the recognition result of the first speech recognition unit 102 is different from the recognition result of the second speech recognition unit 106, the recognition result of the first speech recognition unit 102 is rejected and the recognition result of the second speech recognition unit 106 is used. You may make it employ | adopt. FIG. 12 shows a learning example sentence when the recognition result of the second speech recognition unit 106 is added to the learning example sentence, and FIG. 13 shows a language model at this time. FIG. 14 shows logarithmic probabilities for correct word sequences and N-gram orders applied when the recognition result of the first speech recognition unit 102 is learned and when the recognition result of the second speech recognition unit 106 is learned. A comparison is shown. The total log probability when learning the recognition result of the second speech recognition unit 102 is −2.5464, and the language likelihood is improved as compared with the case where the recognition result of the first speech recognition unit 106 is learned.

  As described above, the second language model is input by the word sequence of the recognition result of the first language model by including the N-gram adding unit that additionally mixes the word sequence of the recognition result of the first language model with the second language model. It is possible to improve the speech recognition performance of the speech recognition apparatus by adapting to speech and improving the coverage of word chains appearing in the input speech of the second language model.

  In this embodiment, the first speech recognizer and the second speech recognizer are separate, but the same single speech recognizer may be used. Further, although the first language model and the second language model are different language models stored in the first language model storage unit and the second language model storage unit, respectively, one language stored in the same language model storage unit It may be a model. At this time, for example, if the utterance is “speech recognition” and the original language model has only one gram of “speech” and “recognition”, “speech recognition” is learned and “<s> speech”, Since two grams of “voice recognition” and “recognition </ s>” are learned, the voice recognition performance can be improved.

Embodiment 2. FIG.
In the first embodiment, one speech recognition result is added to the mixed language model. Next, two speech recognition results obtained from two speech recognizers are added to the mixed language model. The form of is shown.
FIG. 15 is a block diagram of a speech recognition apparatus according to Embodiment 2 of the present invention. The speech recognition apparatus according to Embodiment 2 includes a speech input unit 101, a first speech recognition unit 102, a first language model storage unit 103, a first acoustic model storage unit 104, an N-gram addition unit 105b, and a second speech recognition unit 106. , A second language model storage unit 107b, a second acoustic model storage unit 108, a third speech recognition unit 112, a third language model storage unit 113, and a third acoustic model storage unit 114. The portions denoted by the same reference numerals as those in the first embodiment are the same as those in the first embodiment, and thus the description thereof is omitted.

  The third speech recognition unit 112 is stored in the third language model and third acoustic model storage unit 114 stored in the third language model storage unit 113, similarly to the first speech recognition unit 102 and the second speech recognition unit 106. The speech recognition process is performed with reference to the third acoustic model. The third language model stored in the third language model storage unit 113 is a language model learned from specialized learning example sentences in a specific field similar to the second language model of the first embodiment.

  The N-gram adding unit 105b uses the second language model stored in the second language model storage unit 107b based on the recognition result word sequence of the first speech recognition unit 102 and the recognition result word sequence of the third speech recognition unit 112. To add N grams. The second language model stored in the second language model storage unit 107b is a language model referred to by the second speech recognition unit 106 as in the first embodiment. However, this language model is a language model for storing the N-gram added by the N-gram adding unit 105b. In the initial state, the N-gram is not learned and the learning example sentence is not stored.

  Next, the operation of the speech recognition apparatus according to the second embodiment will be described focusing on differences from the first embodiment. FIG. 16 is a processing flow of the speech recognition apparatus according to the second embodiment. The feature of this embodiment is the processing of ST208 and ST204b shown in FIG. Other processes are the same as those in the first embodiment. In the process of ST208, the third speech recognition unit 112 is stored in the third acoustic model storage unit 113 and the third acoustic model storage unit 114 by the same processing as the first speech recognition unit 102. A word series (third speech recognition result) of the recognition result is obtained with reference to the third language model. The word sequence of the recognition result by the first speech recognition unit 102 and the word sequence of the recognition result by the third speech recognition unit 112 are output to the N-gram adding unit 105b.

  The N-gram adding unit 105b creates a learning example sentence from the received word sequence of the recognition result by the first speech recognition unit 102 and the word sequence of the recognition result of the third speech recognition unit 112 (ST204b). This learning example sentence is stored, and the stored learning example sentence is updated in the subsequent speech recognition of another voice input.

  Similar to the first embodiment, a specific operation will be described by taking as an example a case where a voice “I'm going to be here” is input. The recognition result of the first speech recognition unit 102 is a word sequence similar to that shown in FIG. Further, the recognition result of the third speech recognition unit 112 is as shown in FIG. Based on these, the N-gram adding unit 105b creates a learning example sentence. FIG. 18 shows a learning example sentence created.

  The N-gram adding unit 105b obtains an N-gram probability from the learning example sentence shown in FIG. 18, and adds the N-gram shown in FIGS. 19 and 20 to the second language model. Next, the second speech recognition unit 106 obtains a correct speech recognition result as in the first embodiment by performing speech recognition with reference to the second language model having the newly learned N-gram. be able to.

  As described above, since the mixed language model is generated from the word sequence output from the first speech recognizer and the word sequence output from the third speech recognizer, without impairing the effect of improving the coverage ratio, The mixed language model can be made compact. This is particularly effective when the second speech recognizer is implemented in a small device such as a mobile terminal.

  In the second embodiment, the recognition result of the first speech recognition unit that performs speech recognition using the first language model and the recognition result of the third speech recognition unit that performs speech recognition using the third language model are used. Although the mixed language model is updated based on the recognition result, more language models and voice recognition units may be provided, and the mixed language model may be updated by adding the recognition results of those voice recognition units. Further, as shown in the first embodiment, the second language model may be updated based on the recognition result of the second speech recognition unit.

Embodiment 3 FIG.
In the second embodiment, the second language model is generated by mixing the recognition result word strings output by the respective speech recognition units with the same weight when adding N-grams to the second language model. In this embodiment, weighting (mixing weight) is performed for each voice recognition unit, one or more second language models with varying mixing weights are generated, and the maximum likelihood recognition result among all combinations is output. To do.

  Since the basic configuration is the same as that of the second embodiment, the description will focus on the generation of a weighted mixed language model and the comparison of recognition results using a plurality of mixed language models. In the following description, a case where three mixed language models are provided will be described as an example. However, the number of mixed language models may be one or two, or may be four or more. However, when there is only one mixed language model, processing for comparing recognition results is not necessary.

FIG. 21 shows the configuration of the speech recognition apparatus of this embodiment in the first speech recognition unit 102, the third speech recognition unit 112, the N-gram addition unit 105d, the second language model storage unit 107d, and the second speech recognition unit 106d. It is the functional block diagram shown paying attention. In FIG. 21, λ (0 ≦ λ ≦ 1) is a coefficient for weighting the recognition result of the first speech recognition unit 102 and the recognition result of the third speech recognition unit 112. Here, λ is a weight for the recognition result of the first speech recognition unit 102, and (1-λ) is a weight for the recognition result of the third speech recognition unit 112.
Note that the value of λ is determined in advance corresponding to each second language model. Here, λ = 2/3, λ = 1/2, and λ = 1/3, which correspond to the second language model A, the second language model B, and the second language model C shown in FIG. 21, respectively. And

Next, the operation will be described. The N-gram adding unit 105d creates weighted learning example sentences based on λ and 1-λ, and creates a second language model A, a second language model B, and a second language model C based on the respective learning example sentences. To do.
The learning example sentence in the case of λ = 1/2 is the same as the learning example sentence shown in FIG. When λ = 2/3, as shown in FIG. 22, a learning example sentence is generated so that the word sequence of the recognition result of the first speech recognition unit is twice the word sequence of the recognition result of the third speech recognition unit. To do. On the other hand, when λ = 1/3, learning is performed such that the word sequence of the recognition result of the third speech recognition unit is twice the word sequence of the recognition result of the third speech recognition unit, as shown in FIG. Generate example sentences. In this way, a learning example sentence including the word sequence of the recognition result of the first speech recognition unit and the word sequence of the recognition result of the third speech recognition unit is generated so as to have a ratio equivalent to λ: 1−λ, and corresponding to each. The second language model A, the second language model B, and the second language model C are created.
By creating the second language model in this way, the probability corresponding to the N gram added to the second language model can be weighted corresponding to the speech recognition unit that outputs the example sentence that is the basis of the N gram.

The second speech recognition unit 106d refers to each of the second language model A, the second language model B, and the second language model C in the same process as in the first and second embodiments, and selects a word sequence that is a recognition result. These word sequences are compared using Equation 5 below, and the most likely one is output as a recognition result word sequence. In Equation 5, h _A , h _{B, and} h _C are word sequences obtained by speech recognition processing referring to the second language model A, the second language model B, and the second language model C, respectively, and AC (h _A ), AC (h _B ), AC (h _C ) are acoustic likelihoods for the word sequences h _A, h _B , h _C , and LM (h _A ), LM (h _B ), LM (h _C ) are the word sequences h _A, h _B, the language likelihood for h _C, is n _a, n _B, n _C each word sequence h _a, h _B, the number of words h _C. LM_WEIGHT is a language model weight, and LM_PENALTY is a word delivery penalty.

By doing so, the coverage of the word chain appearing in the input speech is improved, the speech recognition performance of the speech recognition device is improved, and a speech recognition result by an appropriate language model mixing ratio is obtained. is there.
In the first embodiment, weighting can also be performed when the word sequence of the recognition result of the first speech recognition unit is added to the second language model.

  In the embodiment described above, the maximum likelihood solution is an object to be mixed, but an N best solution for selecting a plurality of candidates may be used. Further, the mixed language model may be updated based on the result of morphological analysis of the word series recognized by the first speech recognition unit, the second speech recognition unit, and the third speech recognition unit. The morpheme units may be different between the first language model and the second language model or the third language model.

101 speech input unit, 102 first speech recognition unit, 103 first language model storage unit, 104 first acoustic model storage unit, 105, 105b, 105c, 105d N-gram addition unit, 106, 106d second speech recognition unit, 107 107b, 107d Second language model storage unit, 108 Second acoustic model storage unit, 109 Display unit, 112 Third speech recognition unit, 113 Third language model storage unit, 114 Third acoustic model storage unit

Claims (10)

A second language model storage unit for storing a mixed language model,
A second speech recognition unit that recognizes an input speech signal and outputs a second speech recognition result using the mixed language model;
A first speech recognition unit for recognizing an input speech signal using a first language model and outputting a first speech recognition result;
The Symbol before by the first speech recognition unit first speech recognition result is input, to create a N-gram based on the word sequence of the first speech recognition result, which is stored in the second language model storage unit An N-gram adding unit that adds an N-gram created based on the word sequence of the first speech recognition result to the mixed language model;
A speech recognition apparatus comprising: The N-gram adding unit receives a third speech recognition result of the speech signal from a third speech recognition unit different from the first and second speech recognition units, and a word sequence of the first speech recognition result wherein said creating a N-gram based on the word sequence of the third speech recognition result, the previous SL said first word sequence of speech recognition results in the second language model storage unit to said stored mixed language model No. The speech recognition apparatus according to claim 1, wherein an N-gram created based on the word sequence of the speech recognition result of 3 is added. The speech recognition apparatus according to claim 1, further comprising a first language model storage unit that stores the first language model. A third language model storage unit for storing a third language model;
The third speech recognition unit for recognizing the speech signal using the third language model and outputting the third speech recognition result;
The speech recognition apparatus according to claim 2 , further comprising: The N-gram additional unit, before SL to create the N-gram based on the second word sequence of the speech recognition result of the second speech recognition section, the word sequence of the second speech recognition result to the mixed language model based speech recognition apparatus according to any one of claims 1 to 4, characterized by adding N-gram that was created. The speech recognition apparatus according to any one of claims 1 to 5 , wherein the N-gram adding unit further updates the probability of the N-gram stored in the mixed language model. The N-gram adding unit weights the first speech recognition result and the third speech recognition result with weights corresponding to the first speech recognition unit and the third speech recognition unit , respectively, to the mixed language model . The speech recognition apparatus according to claim 2 , wherein a probability of N grams to be added is calculated. The second language model storage unit stores a plurality of the mixed language models,
The N-gram adding unit performs the weighting based on a predetermined combination of the weight for the first speech recognition result and the weight for the third speech recognition result corresponding to each of the plurality of mixed language models. And calculating a probability of N grams to be added to each of the plurality of second language models,
The second speech recognition unit performs speech recognition with reference to each of the plurality of mixed language models, selects one of the obtained speech recognition results based on the likelihood of the speech recognition result, and selects the first 2 speech recognition result,
The speech recognition apparatus according to claim 7 . A first speech recognition procedure in which the first speech recognition unit performs speech recognition of speech input with reference to the first language model;
A second speech recognition procedure in which a second speech recognition unit performs speech recognition of the input speech with reference to a mixed language model;
An N-gram adding unit creates an N-gram based on the word sequence of the speech recognition result by the first speech recognition procedure, and creates the N-gram created based on the word sequence of the first speech recognition result in the mixed language model. N-gram addition procedure to be added,
A speech recognition method comprising: A third speech recognition unit that performs speech recognition of the input speech with reference to a third language model,
The N-gram addition procedure creates an N-gram based on the word sequence of the speech recognition result by the first speech recognition procedure and the word sequence of the speech recognition result by the third speech recognition procedure, and adds the N-gram to the mixed language model. claim 9, characterized in that to add a N-gram that was created based on the word sequence of the speech recognition result by the first of the third speech recognition procedure and the word sequence of the speech recognition result by the speech recognition procedure Voice recognition method.

Images