JP4852448B2 - Error tendency learning speech recognition apparatus and computer program - Google Patents

Description

  The present invention relates to an error tendency learning speech recognition apparatus and a computer program.

The linguistic likelihood of the recognition result output by the speech recognition apparatus is evaluated by a statistical language model. Many of the statistical language models often use a word n-gram model for obtaining a probability of a word connected to a history from a word history (word string) obtained at a certain point in a speech recognition apparatus. This statistical language model is described in Non-Patent Document 1, for example.
Since the statistical language model is learned from data composed only of positive examples, that is, text correctly written as Japanese, linguistic errors in speech recognition cannot be eliminated. Therefore, if erroneous recognition is included in a certain utterance content, there is a high possibility that a similar utterance content is erroneously recognized.

Therefore, the speech recognition errors are reduced by collecting texts of topics that are close in time or related to speech contents that are traditionally being used for speech recognition and adopting a statistical language model. Things have been done. Non-Patent Document 2 describes a technique for improving the recognition rate by increasing the appearance probability of a word included in a recognition result based on information that “words that appeared in the past are easy to use again”. Yes. Non-Patent Document 3 describes a method of selecting a recognition result based on a posteriori probability and using it for learning a statistical language model.
Kitakenji, “Probabilistic Language Model”, The University of Tokyo Press, 1999, p. 57-62 Kuhn, R. De Mori, “A Cache-Based Natural Language Model for Speech Recognition”, IEEE Trans Pattern Analysis and Machine Intelligence (IEEE) On Pattern Analysis and Machine Intelligence), 1990, vol. 12, no. 6, p. 570-583 Riccaridi and D. Hakkani-Tur, “Dynamic Learning: Theory and Applications to Automatic Speech Recognition”, IEEE Transformer Speech and Audio ( IEEE Trans. On Speech and Audio), 2005, vol. 13, no. 4, p. 504-511

  In the method based on Non-Patent Document 2, the recognition result does not necessarily include the correct word, so there is a possibility of increasing the appearance probability of the error word included in the recognition result. Therefore, the improvement of the recognition rate is limited. In the method based on Non-Patent Document 3, it is possible to learn a language model by giving a correct answer manually, but it is not possible to learn an error tendency of the speech recognition apparatus.

On the other hand, in practical real-time speech recognition apparatuses, in many cases, the output result of the speech recognition apparatus is manually corrected. Since a correct answer for the input speech is always obtained, an error tendency of the recognition result can be obtained by comparing the recognition result output from the speech recognition apparatus with the correct answer. That is, it is possible to identify an error in a word or a word string as a recognition result output from the speech recognition apparatus. When voices having similar utterance contents are input, if the error tendency is learned, the possibility of mistakes is reduced.
In other words, by detecting the error tendency of the speech recognition apparatus from the correct answer and the recognition result and feeding back this information to the speech recognition apparatus, it is possible to reduce speech recognition errors and reduce the load on the correction operator.

  The present invention has been made in consideration of such circumstances, and its purpose is to statistically learn the error tendency of the recognition result by using the recognition result by the speech recognition apparatus and the correct answer corrected by the recognition result. It is an object to provide an error tendency learning speech recognition apparatus and a computer program capable of reducing recognition errors in speech recognition.

  The present invention has been made to solve the above problem, and recognizes an input speech, outputs a plurality of correct answer candidates, selects a speech recognition result from the output correct candidates, and the speech Receiving a correction input for the voice recognition result selected by the recognition means, correcting the voice recognition result and outputting a correct answer to the input voice; a plurality of correct answer candidates output by the voice recognition means; Error tendency learning means for statistically analyzing the tendency of recognition errors from correct answers output by the correction means, and the speech recognition means corrects the tendency of recognition errors analyzed by the error tendency learning means. The error tendency learning speech recognition apparatus is characterized in that an error in selecting a speech recognition result is corrected using an error correction model.

  Further, the present invention is the error tendency learning speech recognition apparatus described above, wherein the error correction model includes the speech recognition correct candidate and the word included in the correct answer, the part of speech or semantic information of the word, the preceding and following words. It is statistically calculated so as to maximize the probability that the correct answer is selected from the speech recognition result based on one or more pieces of information in the column or the dependency.

  Further, the present invention is the error tendency learning speech recognition apparatus described above, wherein the error tendency learning means includes a plurality of correct answer candidates output by the speech recognition means for new input speech, and the correction means. The error correction model is updated by statistically analyzing the tendency of recognition errors from the correct recognition result of the input speech that is output, and the speech recognition means is updated by the error tendency learning means. Is used to correct an error in selecting a speech recognition result.

  Further, the present invention is the error tendency learning speech recognition apparatus described above, wherein the speech recognition means outputs a speech recognition result of the input speech in real time.

  Further, the present invention provides a computer used as an error tendency learning speech recognition device, speech recognition step for speech recognition of input speech and outputting a plurality of correct answer candidates, and selecting speech recognition results from the output correct answer candidates; Receiving a correction input for the voice recognition result selected in the voice recognition step, correcting the voice recognition result and outputting a correct answer to the input voice; and a plurality of correct answer candidates output in the voice recognition step And an error tendency learning step that statistically analyzes the tendency of recognition errors from the correct answer output in the correction step. In the speech recognition step, the recognition error analyzed in the error tendency learning step is analyzed. A process for correcting errors in selecting speech recognition results using an error correction model for correcting trends To be executed by the computer, a computer program, characterized in that.

  According to the present invention, the correct candidate and the correct answer output as a result of speech recognition are statistically learned from the tendency of speech recognition errors, and the recognition error in speech recognition is performed using the statistical model obtained as a result of learning. It is possible to improve the recognition rate by eliminating the above, and it is possible to reduce the load on the correction operator.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram for explaining an outline of an error tendency learning speech recognition apparatus according to an embodiment of the present invention.
Assume that N correct answer candidates are obtained for a certain input speech in the speech recognition device provided in the error tendency learning speech recognition device. These N correct answer candidates are arranged in the order in which they are considered to be likely from among the outputs of the speech recognition apparatus, and the correct answer candidates are shown in FIG. 1 “family / of / resume / of / schedule”, correct candidate 2 “family / of / lowest / of / schedule”, and so on. Further, it is assumed that a correct answer in which errors of insertion, replacement, and omission are corrected manually is obtained based on the first correct answer candidate (correct answer candidate 1) among N answers. Here, the correct answer obtained is “family / of / reunion / of / date”.

Focusing on the portion surrounded by the dotted line in FIG. 1 (symbol A), the correct answer is “reunion”, while the correct answer candidate 1 is “restart” and the correct answer candidate 2 is “lowest”. These are all incorrect. The tendency of errors output by speech recognition results is that when the context “family / no” is given, the speech recognition system selects “resume” or “lowest” without selecting “reunion”. It means to end.
Error tendency learning is to prevent the speech recognition system from selecting “Resume” or “Lowest” when the relevant context is given. Statistically, “Resume” or “Lowest” appears. It is to make it harder or to make “reunion” more likely to appear. The error tendency is evaluated based on the “correctness” of the correct answer candidates, and is given by statistical means.
Below, the block diagram and operation | movement of the error tendency learning speech recognition apparatus by one embodiment of this invention are demonstrated.

  FIG. 2 is a diagram showing an overall configuration of the error tendency learning speech recognition apparatus 1 according to the embodiment of the present invention. The speech recognition apparatus 10 receives the input speech 40 and outputs a correct answer candidate list 60 that is data indicating N correct answer candidates. In general, N is about 200 to 300. The speech recognition apparatus 10 includes an acoustic model 20 that is acoustic model data stored in the acoustic model storage unit 21, a language model 30 that is language model data stored in the language model storage unit 31, and an error correction model storage unit 91. An error correction model 90 that is error correction model data stored therein is used to calculate a score for determining the likelihood of each correct answer candidate, and the input speech 40 is likely based on the calculated score. The correct answer candidate is determined. That is, the speech recognition apparatus 10 searches for a word string having the highest sum of scores of each model using the acoustic model 20, the language model 30, and the error correction model 90, and determines the top N correct answer candidates of the score. The correct candidate list 60 is output. Any existing model can be used for the acoustic model 20 and the language model 30.

  Here, assuming that one of the correct answer candidates is w, the speech recognition apparatus 10 calculates the score g (w) as in the following (Equation 1).

In the above, f ₀ (w) is the score of the acoustic model, and f ₁ (w) is the score of the language model. When the input speech is x, the acoustic score is a score obtained by converting the probability P (x | w) obtained from the statistical acoustic model into a logarithm. The language score is a score obtained by converting the probability P (w) obtained by the statistical language model into a logarithm. This is based on the following. That is, from the Bayes' theorem, the posterior probability P (w | x) = P (x | w) · P (w) / P (x) that the correct candidate w is obtained when the input speech x is generated. Since P (x) has a probability of 1, if the logarithm of both sides is taken, the right side is logP (x | w) + logP (w), and the weights of these terms are λ ₀ and λ ₁ , respectively.
Λ ₀ and λ _{1 in} (Equation 1) are constants and are determined in advance. G _ec (w) in (Equation 1) is the score of the error correction model, and it is determined that the higher the g (w) score, the more likely it is.

In the correction device 50, the first recognition result among the N correct answer candidates shown in the correct answer candidate list 60 output from the speech recognition device 10 is manually input using an input unit (not shown). In response to the correction input, correct data 70 indicating the correct speech recognition result generated based on the correction is output.
The error tendency learning device 80 learns the error tendency in the speech recognition device 10 from the N correct answer candidates indicated by the correct answer candidate list 60 and the correct answer indicated by the correct answer data 70. A method of creating an error correction model in the error tendency learning device 80 will be described with reference to FIG.

  FIG. 3 is a diagram illustrating an operation flow of the error tendency learning device 80. The creation of an error correction model in the error tendency learning device 80 is composed of two procedures: a weight initialization step (S100) and a weight update step (S110). The purpose of the weight initialization step (S100) is to estimate the weight of the error correction model from a large number of correct answer candidates and correct answers accumulated in advance. The purpose of the weight update step (S110) is to update the error correction model obtained in the weight initialization step (S100) using the correct answer candidates and correct answers newly input from the speech recognition apparatus 10.

The weight initialization step (S100) will be described with reference to FIG.
FIG. 4 is a functional block diagram in the error tendency learning device 80 that operates in the weight initialization step.
The score of the error correction model is given as g _ec (w) in (Equation 1), which is defined as in (Equation 2) below.

The error correction model is defined as a linear sum of the functions f _i (w) (i = 2, 3,..., I) in the above equation. Here, f _i (w) (i = 2, 3,..., I) is called a feature function, and is a rule for expressing an error tendency in the recognition result (correct answer candidate) w. Held in. The feature function returns a real number other than 0 when a certain event (here, the correct answer candidate w) has the tendency (feature). As an example, the feature function represents the number of observations of the tendency (feature). These rules are determined based on grammatical information such as a word in a correct answer candidate, corresponding part of speech and semantic information, a word string (context), and dependency. λ _i is a weight related to the feature function, and indicates how important f _i is. λ _i is obtained by a learning procedure described later.
Examples of rules used for feature functions include the following.

Rule 1: Number of word strings “family / no / reunion” included in w (denoted as f ₂ )
Rule 2: Number of clauses in w that are clauses that include “Family” are clauses that contain “Reunion” (set as f ₃ )

The feature function of rule 1 represents the appearance frequency of the context “family / of / reunion” in the correct answer candidate w. The feature function of rule 2 returns the number of correct answers w if rule 2 holds, and returns 0 otherwise.
The feature extraction unit 130 uses the above rules using the correct answer candidate / correct answer 120, that is, the large number of correct answer candidates indicated in the correct answer candidate list 60 output from the speech recognition apparatus 10 in advance and the correct answer output by the correction apparatus 50. Extract features that match.
Then, the initial weight learning unit 140 determines the weight based on the extracted feature function. In order to determine the weight, for example, an objective function such as the following (formula 3) is considered.

Here, w _{m, 0} indicates the correct sentence for the m-th input speech 40, and w _{m, n} indicates the n-th correct candidate for speech recognition for the m-th input speech 40. Objective function above, the input speech x _m conditional probability of correct sentence for q | based on the sum of the log-likelihood when determined as (w x) the following (Equation 4).

However, Z (x _m ) in the above equation is a normalization term, and is expressed as in the following (Equation 5).

That is, in (Equation 3), the ratio of how much the score g (w) of the correct answer occupies in the total of the scores g (w) of the correct answer candidates 1, 2,. (Conditional probability shown in (Equation 4)) is calculated for each input speech x _m (m = 1 to M), and a logarithm is taken (log likelihood), and all M input speeches x _m are added. Loss function. The exp part is for reducing the logarithm. The conditional probability that is the true number of log is 1 or less because of the ratio to the whole, and the value of log is negative. Therefore, the negative sign is attached to the whole to return to a positive value. Since the value of log becomes closer to 0 as the ratio becomes closer to 1, if the L _log is made the smallest, the ratio of correct answers to the whole increases.

Further, as another objective function, a function L _acc based on the expected value of the word correct accuracy for the m-th sentence may be defined as in the following (formula 6).

In the above equation, A _cc (w _{m, n} ) represents the word correct accuracy for the correct answer candidate w _{m, n} . The correct word accuracy is obtained by (number of correct words−number of erroneous insertion words) / (total number of words) × 100.
The number of erroneous insertion words is the number of words that have been inserted or replaced. For example, in the example shown in FIG. 1, the number of words is 5, and in the case of a correct answer, the word accuracy is 100%. In addition, since the recognition results 1 and 2 replace one word “restart” or “lowest” with “reunion”, the number of erroneous insertion words is “1”, and the word accuracy is (5-1) / 5 = 80. %. Also, for example, when the correct answer is “Reunion schedule with family”, “to” is inserted into recognition result 1 and “resumption” is replaced with “reunion”, so the number of erroneous words inserted becomes 2, and the recognition result The word accuracy of 1 is (5-2) / 5 = 60%.

In other words, in (Equation 6), for each input speech x _m (m = 1 to M), the expected value of the correct conditional probability is calculated and a logarithm is taken, and these M input speech x as learning data are used. All _m values are added to obtain a loss function. The log's true number is less than or equal to 1 and is returned to a positive value by adding a negative sign to the whole. However, as the probability that a correct answer appears is closer to 1, the value of log becomes closer to 0. Therefore, if L _acc is _minimized , the expected value at which the correct answer appears as a whole increases.

As described above, the error correction model reflecting the error tendency has a weight that minimizes the objective function L _log or L _acc in the above equation. In order to obtain a weight that minimizes the above objective function, for example, a quasi-Newton method is used. In the quasi-Newton method, an appropriate initial value is given to generate the next value close to the solution, and the value close to the next solution is repeatedly generated from that value, and finally converges to the optimal solution. . For details of the quasi-Newton method, see the document “WH Press et al.,“ Numerical Recipes in C ”, (Translation) Tankei et al., Pp.313-314, 1993.”
The initial weight learning unit 140 outputs the initial error correction model 150 having the weight λ _i obtained by the above procedure. The initial error correction model 150 is written in the error correction model storage unit 91 as an initial value of the error correction model 90.

Next, the weight update step (S110) in FIG. 3 will be described with reference to FIG.
FIG. 5 is a block diagram showing the configuration of the error tendency learning device 80 that operates in the weight update step.
A feature of speech recognition for news and the like is that speech recognition results are obtained and accumulated every moment. Therefore, it is necessary to update the initial error correction model using newly obtained correct answer candidates and correct answers.

The weight update unit 170 newly performs the correct answer candidate and correct answer 160 obtained as the output of the speech recognition apparatus 10 after executing the previous weight initialization step (S100), and the previous weight initialization step (S100). Using the obtained initial error correction model 150 as an input, the weight λ _i in the current initial error correction model 150 is updated.
The new weight is obtained based on the steepest descent method. The steepest descent method is a method of searching for the minimum value of a function from the first-order derivative (slope) of the function. For details of the steepest descent method, refer to the document “RO Duda, PE Hart and DG Stork,“ Pattern Classification (2nd edition) ”, pp.223-227, 2001.
The update formula of the weight λ _i by the steepest descent method is as follows.

Here, η is a constant, and a predetermined value is used. Further, the objective function L (Λ) is based on (Equation 3) or (Equation 6), the newly obtained correct answer candidate and the correct answer 170, that is, the correct answer candidate newly output from the speech recognition apparatus 10 and the correction L _log or L _acc calculated using the correct answer output by the device 50.
The weight update unit 170 updates the error correction model 90 by replacing the obtained λ _i ′ with the original λ _i . The speech recognition apparatus 10 selects the correct sentence from the correct answer candidates using the acoustic model 20, the language model 30, and the error correction model 90.

The procedure for selecting the correct sentence will be described with reference to FIG. FIG. 6 is a block diagram showing the configuration of the speech recognition apparatus 10 that operates when a correct sentence is selected.
The speech recognition apparatus 10 generates N correct answer candidates 180 from the input speech 40. The score calculation unit 190 uses the acoustic model 20, the language model 30, and the error correction model 90 to calculate a score for each correct answer candidate 180 according to (Equation 1).
Subsequently, the correct answer candidate sorting unit 200 rearranges the correct answer candidates 180 in descending order of score according to the score obtained by the score calculating unit 190. As a result of the rearrangement in the correct answer candidate sorting unit 200, the correct answer candidate 180 that is ranked first is set as the speech recognition result 210 and is output from the speech recognition system 10.
Note that, by using a normal general personal computer or the like, the speech recognition apparatus 10 can perform the processing from the input of the input speech 40 to the output of the speech recognition result 210 described above in real time.

Below, speech recognition was performed on news (1,298 sentences) broadcasted in the past based on the above embodiment, and word accuracy was obtained.
Table 1 shows the result of comparison with the conventional method of re-scoring using the trigram language model.

The embodiment described above has the following effects.
(A) Reducing similar recognition errors and reducing the burden on corrective operators in applications such as real-time speech recognition systems.
(B) Since the error tendency model increases the accuracy by repeating the sequential learning times, the recognition rate of the speech recognition system improves with time.

In the above description, both the coefficient λ ₀ of the score of the acoustic model and the coefficient λ ₁ of the score of the speech recognition model are changed, but these are not changed, and the coefficient λ _i (i = 2 to I) of the feature function is not changed. Only the change may be made.

  The error tendency learning speech recognition apparatus 1 described above has a computer system inside. The process of operation of each device of the error tendency learning speech recognition device 1 is stored in a computer-readable recording medium in the form of a program, and the computer system reads and executes this program, whereby the above processing is performed. Done. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

It is a figure for demonstrating the outline | summary of the error tendency learning speech recognition apparatus by one embodiment of this invention. It is a functional block diagram of the error tendency learning speech recognition apparatus by the embodiment. It is a figure which shows the operation | movement flow of the error tendency learning apparatus by the embodiment. It is a block diagram which shows the structure of the error tendency learning apparatus which concerns on the weight initialization step by the embodiment. It is a block diagram which shows the structure of the error tendency learning apparatus which concerns on the weight update step by the embodiment. It is a figure which shows the structure of the speech recognition apparatus by the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Error tendency learning voice recognition apparatus 10 ... Voice recognition apparatus (voice recognition means)
20 ... Acoustic model 30 ... Language model 50 ... Correction device (correction means)
80 ... Error tendency learning device (error tendency learning means)
DESCRIPTION OF SYMBOLS 90 ... Error correction model 130 ... Feature extraction part 140 ... Initial weight learning part 170 ... Weight update part 190 ... Score calculation part 200 ... Correct candidate sorting part

Claims (4)

Speech recognition means for speech recognition of input speech, outputting a plurality of correct answer candidates, and selecting speech recognition results from the output correct answer candidates;
Correction means for receiving a correction input to the voice recognition result selected by the voice recognition means, correcting the voice recognition result, and outputting a correct answer to the input voice;
Storage means for storing an error correction model defined by a feature function representing an error tendency of a correct candidate and its weight;
Error tendency learning that creates the error correction model by determining the weight based on a plurality of correct answer candidates output by the speech recognition means and the correct answer output by the correction means, and stores the error correction model in the storage means Means and
The speech recognition means selects a speech recognition result from correct candidates based on an acoustic model, a language model, and the error correction model ,
The error tendency learning means determines the weight so that a correct score calculated based on each model is larger than a score of each candidate other than the correct answer among the correct candidates. Trend learning speech recognition device. Speech recognition means for speech recognition of input speech, outputting a plurality of correct answer candidates, and selecting speech recognition results from the output correct answer candidates;
Correction means for receiving a correction input to the voice recognition result selected by the voice recognition means, correcting the voice recognition result, and outputting a correct answer to the input voice;
Storage means for storing an error correction model defined by a feature function representing an error tendency of a correct candidate and its weight;
Error tendency learning that creates the error correction model by determining the weight based on a plurality of correct answer candidates output by the speech recognition means and the correct answer output by the correction means, and stores the error correction model in the storage means Means and
The speech recognition means selects a speech recognition result from correct candidates based on an acoustic model, a language model, and the error correction model,
The error tendency means such that said probability of speech recognition result coincides with the correct answer is maximized, tend training speech recognition apparatus Ri erroneous you characterized by determining the weight. The voice recognition means, error tendency speech recognition apparatus according to claim 1 or claim 2, characterized in that for outputting a speech recognition result of the input speech in real time. In a computer used as an error tendency learning speech recognition device,
A speech recognition step for recognizing input speech to output a plurality of correct answer candidates and selecting a speech recognition result from the output correct answer candidates;
A correction step of receiving correction input for the voice recognition result selected in the voice recognition step, correcting the voice recognition result and outputting a correct answer to the input voice;
Based on the plurality of correct answer candidates output by the speech recognition step and the correct answer output by the correction step, an error correction model defined by a feature function representing an error tendency of the correct answer candidate and its weight is selected as the weight. And an error tendency learning step created by determining
In the voice recognition step, based on the acoustic model, the language model, and the error correction model, let the computer execute a process of selecting a voice recognition result from the correct answer candidates ,
In the error tendency learning step, the computer executes the process of determining the weight so that the correct score calculated based on each model is greater than the score of each candidate other than the correct answer among the correct candidates. A computer program characterized by causing

Images