JP6086714B2 - Speech recognition apparatus, error correction model learning method, and program - Google Patents

Description

  The present invention relates to a speech recognition device, an error correction model learning method, and a program.

  For error correction in speech recognition, statistical error correction models obtained as a result of learning by statistically learning the tendency of speech recognition errors using linguistic features from speech and transcriptions (correct sentences) There is a technology for improving the performance of speech recognition by using (see, for example, Non-Patent Document 1). In addition, there is a technique for learning an error correction model from learning data without a correct word string and improving speech recognition performance (for example, see Non-Patent Document 2).

Kobayashi et al., “News speech recognition by discriminative scoring based on word error minimization”, IEICE Journal, vol.J93-D no.5, 2010, p. 598-609 Kobayashi, A., Oku, T., Homma, S., Imai, T. and Nakagawa, S., "Lattice-based risk minimization training for unsupervised language model adaptation", Proc. Interspeech, pp.1453-1456, 2011 .

In speech recognition for broadcast programs and the like, a plurality of continuous utterances are successively recognized, but the content of the utterance being processed by the speech recognition is often related to the utterance content immediately before speech recognition is finished. For example, in a cooking program, if there is an utterance about the introduction of ingredients, it is expected that the utterance about the cooking method will continue thereafter. That is, there is a high possibility that words related to food ingredients and words related to the cooking method co-occur in adjacent utterances. For example, if the utterance of “pick a pork fin” is followed by the utterance of “next salt and pepper”, there is a relationship between “pig fin” and “salt pepper”, and these are co-occurring It will be easy to do.
However, in the conventional error correction model model parameter learning shown in Non-Patent Documents 1 and 2, information such as word co-occurrence between utterances related to the utterance order is not taken into account, so the utterance content is correctly predicted. It is not the optimal model above.
In Non-Patent Document 1, when learning a conventional error correction model, a large amount of speech data and a correct word string that is a transcription of the speech data are required. Estimating a statistically robust model requires a large amount of learning data, but has the disadvantage that the cost of creating a transcript is high.

  The present invention has been made in consideration of such circumstances, and uses the relationship between utterances derived from speech recognition results of utterances that differ in time, and optimizes the error correction model while reducing costs. A recognition device, an error correction model learning method, and a program are provided.

[1] One aspect of the present invention is included in a speech language resource storage unit that stores speech recognition results obtained by speech recognition of speech speech data while retaining the order of speech, and the speech recognition results. A linguistic feature corresponding to the order of utterances is acquired from a word and a word included in the speech recognition result of a past utterance than the speech recognition result, and the acquired linguistic feature is used as the utterance of the past utterance. An error correction model for weighting according to the posterior probability of a speech recognition result, statistically learning a tendency of recognition error of a word based on the weighted linguistic feature, and correcting the tendency of the learned recognition error An error tendency learning unit for generating a speech recognition device.
According to the present invention, the speech recognition apparatus recognizes speech speech data, and includes words included in the obtained speech recognition result, and words included in speech recognition results of past utterances than the speech recognition result. Then, linguistic features corresponding to the order of utterances are extracted. As the speech recognition result of the past utterance, for example, the speech recognition result of the latest past utterance adjacent in time is used. The speech recognition device weights the extracted linguistic features according to the posterior probabilities of speech recognition results of past utterances from which the linguistic features were obtained, and recognizes words based on the weighted linguistic features. The error tendency is statistically learned, and an error correction model for correcting the learned recognition error tendency is generated.
This makes it possible to generate an error correction model suitable for correctly predicting the utterance content using the relationship between utterances derived from the speech recognition results of utterances that differ in time, and to transcribe speech data. Can reduce the cost.

[2] One aspect of the present invention is the speech recognition apparatus described above, wherein the error tendency learning unit includes the same utterance obtained from the linguistic features according to the weighted utterance order and the speech recognition result. The tendency is to statistically learn the tendency of recognition errors of words based on the linguistic features, and to generate an error correction model for correcting the tendency of learned recognition errors.
According to this invention, the speech recognition device extracts linguistic features corresponding to the order of utterances from speech recognition results of utterances that differ in time, and linguistic features in the same utterance are extracted from each speech recognition result. Extract. The speech recognition apparatus statistically learns the tendency of recognition errors of words based on these extracted linguistic features, and generates an error correction model for correcting the tendency of learned recognition errors.
As a result, the speech recognition apparatus uses the linguistic features in the same utterance in addition to the information extracted from the content of the utterances past than the utterance that is the target of speech recognition, to accurately recognize the recognition error. An error correction model to be corrected can be generated.

[3] One aspect of the present invention is the speech recognition apparatus described above, wherein the linguistic feature according to the order of the utterances includes the words included in the speech recognition results and the speech recognition results of the past utterances. The linguistic features in the same utterance are co-occurrence relationships of a plurality of consecutive words in the same utterance obtained from the speech recognition result, It is one or more of co-occurrence relation, syntactic information of words, or semantic information of words.
According to this invention, the speech recognition apparatus is capable of co-occurrence relationships between words obtained from speech recognition results of utterances that differ in time, and word co-occurrence relationships and syntactical relationships in the same utterance obtained from each speech recognition result. Then, the error tendency of the word is statistically learned based on the semantic information, and an error correction model for correcting the tendency of the recognized recognition error is generated.
Thereby, the speech recognition apparatus can generate an error correction model that corrects a recognition error with high accuracy.

[4] One aspect of the present invention is the speech recognition device described above, wherein the error correction model is a language according to the order of the utterances weighted by the posterior probabilities of the speech recognition results of the past utterances. A first feature function based on a characteristic feature, a second feature function based on a linguistic feature in the same utterance, and a feature weight of each of the first feature function and the second feature function. A calculation formula for correcting a score of speech recognition, wherein the error tendency learning unit includes the speech recognition result and the value of the first feature function obtained from the speech recognition result of the past utterance, and the speech The number of word errors obtained by comparing the value of the second feature function obtained from the recognition result with a plurality of the speech recognition results obtained from the same speech data is expressed as the posterior probability of the speech recognition result. With weighted values The feature weight is statistically calculated on the basis of an evaluation value calculated by an evaluation function determined in such a manner, and the error correction model is generated using the calculated feature weight.
According to the present invention, the error correction model includes a feature function representing a linguistic feature according to the order of utterances weighted by a posteriori probability of a speech recognition result of a past utterance, and a linguistic feature within the same utterance. Is a calculation formula for correcting the score of speech recognition based on a feature function representing, and feature weights of these feature functions. The speech recognition apparatus detects a word error obtained by comparing the value of a feature function obtained from speech recognition results of different utterances or the same speech with a plurality of speech recognition results of the same utterance. The feature weight is determined so that the evaluation value calculated by the evaluation function determined using the value weighted by the posterior probability of the speech recognition result is the evaluation value indicating that there is the least recognition error. Generate a modified model.
Thereby, the speech recognition apparatus can learn the recognition error tendency efficiently and generate an error correction model.

[5] One aspect of the present invention is the speech recognition device described above, wherein the input speech data is speech-recognized, and the input is performed using the error correction model generated by the error tendency learning unit. It further comprises a speech recognition unit that corrects an error in selecting a speech recognition result obtained from speech data.
According to this invention, the speech recognition apparatus selects a speech recognition result using an error correction model from among correct answer candidates obtained by speech recognition of speech data.
Thereby, the speech recognition apparatus can obtain a speech recognition result with a good recognition rate.

[6] One aspect of the present invention is included in a speech language resource storage process of storing speech recognition results obtained by speech recognition of speech speech data while maintaining the order of speech, and the speech recognition results. A linguistic feature corresponding to the order of utterances is acquired from a word and a word included in the speech recognition result of a past utterance than the speech recognition result, and the acquired linguistic feature is used as the utterance of the past utterance. An error correction model for weighting according to the posterior probability of a speech recognition result, statistically learning a tendency of recognition error of a word based on the weighted linguistic feature, and correcting the tendency of the learned recognition error And an error tendency learning process for generating an error correction model learning method.

[7] In one aspect of the present invention, a speech language resource storage unit that stores a speech recognition result obtained by speech recognition of speech speech data stored in a computer while maintaining a speech order, and the speech recognition result Linguistic features corresponding to the order of utterances are acquired from the words included in the speech recognition result and the words included in the speech recognition result of the past utterance than the speech recognition result, and the acquired linguistic feature is Weighting according to the posterior probability of the speech recognition result of utterance, statistically learning the tendency of recognition errors of words based on the weighted linguistic features, and correcting the tendency of the learned recognition errors An error tendency learning means for generating an error correction model is a program for functioning as a speech recognition apparatus.

  According to the present invention, it is possible to optimize an error correction model while reducing the cost of transcription of speech data by utilizing the relationship between speeches derived from speech recognition results of speeches that differ in time. Become.

It is a figure which shows the learning method of the error correction model in the speech recognition apparatus by one Embodiment of this invention. It is a figure for demonstrating the example of the feature function according to the order of the speech by the embodiment. It is a functional block diagram which shows the structure of the speech recognition apparatus by the embodiment. It is a figure which shows the whole processing flow of the speech recognition apparatus by the embodiment. It is a figure which shows the model learning process flow of the speech recognition apparatus by the embodiment. It is a figure which shows the learning method of the error correction model by the conventional method.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[1. Overview of this embodiment]
An error correction model that reflects the error tendency of speech recognition has already been devised, but this error correction model is information based on the relationship between utterances that are continuously uttered and temporally adjacent utterance contents. It is not something that uses. Consecutive utterances often include words related to the word used in the previous utterance. Therefore, if such a word connection between adjacent utterances is used in an error correction model, improvement in speech recognition performance is expected.

On the other hand, when generating an error correction model, generally, an error tendency of speech recognition is learned using a correct word string which is a speech and a transcription text thereof. Transcripted text is created manually, but a large amount of learning data is required to obtain the robustness of the statistical model, so there is a drawback that the cost of creating the transcript is high. Further, when a correct word string corresponding to speech cannot be obtained, information on the content of the speech immediately before the speech of interest must be acquired from a plurality of speech recognition results including recognition errors.
Therefore, the speech recognition apparatus according to the present embodiment uses a linguistic feature included in a plurality of speech recognition results for the latest utterance content, and corrects an error correction model in which speech recognition performance is adapted to the utterance content as a correct word string. Learn without and apply it to speech recognition.

[2. Error correction model learning algorithm]
Subsequently, an error correction model learning algorithm applied to the speech recognition apparatus according to the embodiment of the present invention will be described.
As described above, in order to solve the conventional problem, the speech recognition apparatus according to the present embodiment introduces an utterance order relationship into speech data used for learning, and incorporates a relationship between adjacent utterances into an error correction model. The error correction model is learned without the correct word string. The difference between this embodiment and the conventional method is how to handle data when learning an error correction model.

  FIG. 6 is a diagram illustrating an error correction model learning method according to a conventional method. As shown in the figure, in the conventional method, the learning data composed of a plurality of utterances does not preserve the order relationship. In addition, in the conventional method, the correct word string of the utterance is included in the learning data. As described above, the error tendency of speech recognition is learned from learning data that includes a correct word string and does not maintain the order relation.

FIG. 1 is a diagram illustrating an error correction model learning method according to the present embodiment. As shown in the figure, in this embodiment, only the speech recognition result is used as learning data, and the relationship between temporally adjacent utterances is considered in terms of linguistic features in consideration of the order relationship of each utterance in the learning data. And used for learning error correction models. As a result, an error correction model reflecting the relationship between adjacent utterances can be obtained, so that speech recognition performance can be improved as compared with the conventional method. In addition, since a correct word string is not required, the learning cost of the error correction model can be reduced as compared with the conventional method.
As described above, the speech recognition apparatus according to the present embodiment learns an error correction model for correctly predicting the utterance content by using the speech recognition result without the correct word string and the speech recognition result of the immediately preceding utterance.

[2.1 Error correction model of conventional method]
According to Bayes' theorem, when speech input x is given, the most likely word sequence w ^ (“^” represents “hat”) for this speech input x is expressed by the following equation (1). ).

The voice input x and the word string w correspond to, for example, the unit of speech, and P (w | x) is a posterior probability that a word string (sentence hypothesis) w is obtained when the voice input x is given.
P (x | w) is a likelihood indicating acoustic likelihood for the word string w, and the score (acoustic score) is a hidden Markov model (HMM) and a Gaussian mixture distribution (Gaussian Mixture). It is calculated based on a statistical acoustic model (hereinafter referred to as “acoustic model”) typified by Model, GMM). In other words, when an acoustic feature amount is given, a score representing the likelihood of each of a plurality of correct candidate words is an acoustic score.

  On the other hand, P (w) is a linguistic generation probability for the word string w, and the score (language score) is described as a statistical language model (hereinafter, “language model”) such as a word n-gram model. ). In other words, when a word string before or after a speech recognition target word, or both word strings before and after the given word string, a score representing the likelihood of each of a plurality of correct answer word strings is a language score. The word n-gram model gives the occurrence probability of the next word from the history of the word (N-1) based on the statistics of N word chains (N is 1, 2, or 3, for example). It is a model.

  In the following description, HMM-GMM is used for the acoustic model and n-gram is used for the language model.

  When P (x | w) P (w) in Equation (1) is maximum, the logarithm is also maximum. Therefore, in speech recognition, an evaluation function q (w | x) of a word string w, which is a sentence hypothesis (correct answer candidate) when a speech input x is given, based on the Bayes' theorem of the above equation (1). It is defined as the following formula (2).

In formula (2), f _am (x | w) is a logarithmic acoustic score of the word sequence w according to the acoustic model, f _lm (w) is a logarithmic language score of the word sequence w according to the language model, and λ _lm is an acoustic score Is the weight of the language score for.

  When the formula (2) is determined, as shown in the following formula (3), the evaluation function q (w | x) represented by the formula (2) is selected from the set of correct candidate word strings w for the speech input x. ) Is selected as the speech recognition result of speech input x.

  In the error correction model in the conventional method, the maximum likelihood hypothesis is obtained by the following equation (4).

Equation (4) in _{Σ i λ i f i (w} ) is a score reflecting the error tendency of the word sequence w, act as a penalty or reward for the word sequence w. Further, f _i (w) (i = 1,...) Is an i-th feature function, and λ _i is a weight (feature weight) of the feature function f _i (w). The feature function is defined as a function that becomes the number if a linguistic rule is established in a given word string (here, word string w), and is 0 if not established. These rules are, for example, linguistic features such as consecutive words in the same utterance, co-occurrence relationship of two or more words that are not consecutive, syntactic information or semantic information of words. Examples of rules specific feature function f _i in the conventional method, and the like below.

  For example, there are the following (1) and (2) as feature functions based on the co-occurrence relationship of words.

(1) When the word string w includes a continuous word binary set (u, v), a function that returns the number (2) When the word string w includes a non-continuous word binary set (u, v) , A function that returns the number

  For example, the following (3) and (4) are feature functions based on syntax information obtained by replacing each word constituting the word string w with a part-of-speech category (syntax information) such as a noun or a verb. Note that c (•) is a function that maps words to parts of speech.

(3) A function that returns the number of part-of-speech binaries (c (u), c (v)) that are consecutive in the word string w (4) A part-of-speech binary pair that is not consecutive in the word string w u), c (v)), a function that returns the number if it is included

  Alternatively, for example, the following (5) and (6) are feature functions based on semantic information obtained by replacing each word constituting the word string w with a category (semantic category) representing semantic information. is there. The semantic category can be obtained by using a thesaurus stored in a database provided outside or inside the speech recognition apparatus of the present embodiment. Note that s (•) is a function that maps words to semantic categories.

(5) A function that returns the number of semantic category binary groups (s (u), s (v)) that are consecutive in the word string w (6) A semantic category binary group that is not consecutive in the word string w ( a function that returns the number of s (u), s (v))

As described above, an error tendency of speech recognition is expressed as a penalty for a linguistic feature by a feature function and its weight, and is estimated based on an evaluation function that minimizes a word error in learning data. That is, the conventional learning of error tendency is to obtain the weight λ _i of Equation (4) using the learning data.

[2.2 Learning algorithm of error correction model applied to this embodiment]
Now, paying attention to that voice input (speech) x _m word column one of the speech recognition result of (Bunkasetsu) w _m, and _k. Further, to the nearest voice input x _m-1 of the audio inputs x _m, a set of word strings obtained as the speech recognition result of the speech temporally adjacent to G _m-1. In this case, the conditional probability P (w _{m, k} | x _m , G _m-1 ) of the word string w _{m, k} when the speech input x _m and the word string set G _m-1 are given is as follows: Equation (5) is obtained.

However, the derivation of Equation (5) uses the Bayes' theorem and the fact that the set G _m−1 and the speech input x _m are conditionally independent. Further, in the expression (5), the word string w _{m, l} is a plurality of word strings obtained as a voice recognition result of the voice input x _m .
Here, when a speech input x _m and a set of word sequences G _m−1 obtained as speech recognition results of adjacent utterances are given, the most likely word sequence w ^ for the input is Note that Equation (1) is changed to (6).

Here, the conditional probability P (G _m−1 | w) of the set of word strings G _m−1 under the word string w obtained from the latest input speech is assumed as shown in Expression (7). .

  From this assumption, equation (6) becomes the following equation (8).

Γ _j (w, G _m−1 ) (j = 1,...) Is a feature function representing a linguistic feature defined by the word string w and the set of word strings G _m−1 . It is expressed using information between multiple utterances that differ in time. Φ _j is a weight (feature weight) corresponding to γ _j .

FIG. 2 is a diagram for explaining an example of a feature function γ _j of a linguistic feature established between a plurality of utterances that are different in time. In the figure, the speech recognition result of the current utterance of interest is a set of sentence hypotheses w ₁ , w ₂ , w ₃ , and the speech recognition result of the latest utterance is a set of sentence hypotheses u ₁ ,..., U ₄ . It is.
As a linguistic feature established between a plurality of temporally different utterances, the word between the sentence hypothesis of the latest utterance and the sentence hypothesis of the current utterance (words v and z in the figure) as shown below There is co-occurrence.

(Example) The number of words v when the word z is included in the sentence hypothesis of the preceding utterance and the word v is included in the sentence hypothesis of the utterance of interest

A function for obtaining a linguistic feature between utterances as in the above example is represented as h _j (•, •). For example, if the current sentence hypotheses of interest is a sentence hypotheses w ₁ shown in FIG. 2, the word v in Bunkasetsu w ₁ includes a single, sentence hypothesis u ₁ word z is contained one , H _j (w ₁ , u ₁ ) = 1. On the other hand, if the current sentence hypotheses of interest with sentence hypotheses w _3, the word v in Bunkasetsu w ₃ contains two, between the sentence hypothesis u ₄ word z is contained one , H _j (w ₃ , u ₄ ) = 2.

Using this function h _j , the feature function γ _j (•, •) is defined as in the following equation (9).

In Equation (9), p (w _{m−1, n} ) is the posterior probability of the sentence hypothesis w _{m−1, n} that is the nth speech recognition result for the utterance of the immediately preceding input speech. As described above, the feature of this embodiment is that the sentence hypothesis included in the set of speech recognition results of past utterances depends on the posterior probability even when correct word strings for the speech recognition results of past utterances are not given. It is considered that it corresponds to a correct word string, and the value of the feature function obtained from each sentence hypothesis is weighted by the posterior probability and added.
Considering the feature functions of conventional identification language model, the posterior probability of the input speech x _m sentence hypotheses w obtained from a set of sentence hypotheses resulting speech recognition for the input speech x _m-1 of the immediately preceding G _{m- When 1} is obtained, the following equation (10) is obtained.

The model parameter Λ in equation (10) is (λ ₁ , λ ₂ ,...), And the model parameter Φ is (φ ₁ , φ ₂ ,...). Further, Z (Λ, Φ) in the equation (10) is a normalization constant for satisfying the probability condition, and is represented by the following equation (11). Equation (11) word sequence w in 'is a sentence hypotheses of the plurality of speech recognition result obtained by speech recognition from the speech input x _m.

  The learning of the error correction model by the speech recognition apparatus according to the present embodiment is to estimate the model parameters Λ and Φ of the error correction model shown as the exponent part of the exponent function exp on the right side in Equation (10) from the learning data. . As described above, the error correction model of the present embodiment includes a value of a feature function based on a linguistic feature corresponding to an utterance order, a value of a feature function based on a linguistic feature in the same utterance, and these feature functions. It is a calculation formula which corrects the score of voice recognition using the feature weight of.

  Here, when learning data consisting of M utterances to which a correct word string is not given is given, an objective function L (Λ, Φ) for model parameter estimation is expressed by the following equation (12).

m indicates the order of utterances, and N _m is the total number of sentence hypotheses w _{m, 1} , w _{m, 2} ,... generated by speech recognition for the speech input x _m that is the m-th learning data. Also, Χ (w _{m, n} ) is expressed by the following formula (13).

In Expression (13), R (•, •) is a function that returns the edit distance between two word strings, and the sentence hypothesis w _{m, k} is all sentences other than the sentence hypothesis _{m, n} obtained from the speech input x _m. It is a hypothesis. The edit distance between two word strings can be efficiently obtained by dynamic programming. Since the edit distance is equivalent to the number of error words in the speech recognition result for the correct word string (operation of substitution, insertion, omission error), the objective function L (Λ, Φ) in Expression (12) is the expected word. It represents the number of errors. If the model parameter Λ and the model parameter Φ are estimated so as to minimize the objective function L (Λ, Φ), an error correction model that minimizes the number of expected word errors can be obtained. Improvement can be expected. This is because if the model parameters Λ and Φ are estimated so as to minimize the objective function L (Λ, Φ), the recognition error expected for the correct candidate word string is minimized, and the unknown input is different from the learning data. This is because also in speech recognition for speech, recognition errors are similarly minimized by the model parameters Λ and Φ. That is, the objective function of Expression (12) is used as an evaluation function for calculating an evaluation value as to whether or not the model parameters Λ and Φ are appropriate because the recognition error expected for the correct candidate word string is minimized.

Note that P (w _{m, k} | x _m ) in the equation (13) is calculated as the following equation (14).

  In the equation (14), g ^ becomes the following equation (15) from the equation (10).

  If the gradients ΔΛ and ΔΦ related to the model parameters Λ and Φ of the objective function are obtained in order to estimate the model parameters, the following equations (16) and (17) are obtained.

The gradient ΔΛ is (∂L (Λ, Φ) / ∂λ ₁ , ∂L (Λ, Φ) / ∂λ ₂ , ∂L (Λ, Φ) / ∂λ ₃ ,...), And the gradient ΔΦ is (∂L (Λ, Φ) / ∂φ ₁ , ∂L (Λ, Φ) / ∂φ ₂ , ∂L (Λ, Φ) / ∂φ ₃ ,...). Also, Χ (·, ·) was regarded as a constant with respect to the model parameters Λ and Φ.

Assuming that model parameters Λ ^t and Φ ^t are learned by ^t repeated updating, if model parameters Λ ^t−1 and Φ ^t−1 are obtained after the ^t− 1th ^iteration , the following formula ( 18) and Expression (19) are parameter update expressions.

Here, η _Λ and η _Φ are coefficients of the gradient ΔΛ and the gradient ΔΦ obtained by the equations (16) and (17), respectively.
Further, even when a plurality of sets (G _m−1 , G _m−2 ,...) Of speech recognition result sets of adjacent utterances are given, the model parameters Λ and Φ can be learned by performing the same procedure. .

[3. Configuration of voice recognition device]
FIG. 3 is a functional block diagram showing the configuration of the speech recognition apparatus 1 according to an embodiment of the present invention, and only functional blocks related to the invention are extracted and shown.
The speech recognition apparatus 1 is realized by a computer device, and as shown in the figure, a speech recognition unit 11, a feature amount extraction unit 12, an error tendency learning unit 13, a speech recognition unit 14, a speech language resource storage unit 21, an acoustic model A storage unit 22, a language model storage unit 23, and an error correction model storage unit 24 are provided.

  The spoken language resource storage unit 21 stores learning data. The acoustic model storage unit 22 stores an acoustic model. The language model storage unit 23 stores a language model. The error correction model storage unit 24 stores an error correction model.

  The speech recognition unit 11 recognizes speech speech data D1 to generate learning data. The voice data D1 indicates a feature value obtained by performing a short-time spectrum analysis on a voice waveform of an utterance. In this embodiment, audio data is acquired from a broadcast signal. The voice recognition unit 11 associates the voice data D1 with the voice recognition result data D2 obtained by voice recognition of the voice data D1, and writes it in the spoken language resource storage unit 21 as learning data. At this time, the speech recognizing unit 11 also holds the order of utterances when performing speech recognition in the spoken language resource storage unit 21.

The feature amount extraction unit 12 extracts linguistic features in the same utterance and linguistic features according to the utterance order from the speech recognition result data D2 arranged in the utterance order indicated by the learning data. The feature quantity extraction unit 12 outputs feature function data D3 indicating feature functions f _i and γ _j using the obtained linguistic features as rules.

  The error tendency learning unit 13 uses the feature function data D3 output from the feature quantity extraction unit 12 and the learning data stored in the spoken language resource storage unit 21 as inputs, and uses the statistical parameters for model parameters Λ and Φ of the error correction model. To learn. The error tendency learning unit 13 writes the error correction model using the learned model parameters Λ and Φ into the error correction model storage unit 24.

  The speech recognition unit 14 refers to the acoustic model stored in the acoustic model storage unit 22 and the language model stored in the language model storage unit 23, and stores the error correction model stored in the error correction model storage unit 24. Is used to perform speech recognition of the input speech data D4, and speech recognition result data D5 indicating the result is output. The input voice data D4 indicates a feature amount obtained by performing a short-time spectrum analysis on the voice waveform of the utterance.

[4. Processing procedure of voice recognition device]
FIG. 4 is a diagram showing an overall processing flow of the speech recognition apparatus 1 according to the present embodiment. Hereinafter, processing of each step shown in FIG.

[4.1 Step S1]
First, the voice recognition unit 11 acquires the voice data D1 of the program from the broadcast signal and recognizes the voice. The speech recognition unit 11 stores learning data in which the speech data D1 of each utterance is associated with the speech recognition result data D2 indicating the speech recognition result in the speech language resource storage unit 21. At this time, the voice recognition unit 11 stores and stores the order of utterances when voice recognition is performed. M th showing speech recognition result data D2 (m = 1,2, ...) in the training data, and voice input x _m is the m-th audio data D1, obtained by speech recognition of the speech input x _m The sentence hypothesis w _{m, n} (n = 1, 2,...) Is included.

[4.2 Step S2]
The error tendency learning unit 13 extracts a feature function based on linguistic features used for error tendency learning from the learning data stored in the spoken language resource storage unit 21.

First, the error tendency learning unit 13 returns a function of returning the number of consecutive word binaries (u, v) from the speech recognition result data D2 included in the learning data, or a discontinuous word binary set (u, v). Feature functions based on linguistic features within the same utterance, such as continuous words, non-consecutive two or more words, syntactic or semantic information of words, such as functions that return the number of words Are all extracted.
Further, the error tendency learning unit 13 refers to all combinations of the sentence hypothesis w _{m, n of the} correct candidate indicated by the speech recognition result data D2 and the sentence hypothesis w _{m−1, n} of the preceding utterance. As shown in 9), all feature functions based on linguistic features corresponding to the utterance order are extracted.
The error tendency learning unit 13 counts the frequency at which these extracted feature functions appear. The error tendency learning unit 13 obtains a feature function based on a linguistic feature in the same utterance whose counted appearance frequency is equal to or higher than a predetermined threshold, and a feature function based on a linguistic feature according to the order of utterances in step S3. It is determined as a feature function f _i and feature function γ _j used for learning model parameters in the error tendency learning process. The error tendency learning unit 13 outputs the feature function data D3 in which the determined feature functions f _i and γ _j are set to the error tendency learning unit 13.

[4.3 Step S3]
Subsequently, the error tendency learning unit 13 learns model parameters Λ and Φ of the error correction model.
FIG. 5 is a diagram illustrating a processing flow of the error correction model update process executed by the error tendency learning unit 13 in step S3.

(Step S31: Model parameter initialization process)
The error tendency learning unit 13 sets appropriate initial values for the model parameters Λ and Φ. In this embodiment, the initial value is Λ = Φ = 0, and all parameters are set to zero.

(Step S32: Edit distance calculation process)
In order to calculate the objective function of Expression (12), it is necessary to calculate the edit distance between the speech recognition results obtained from the same utterance. Therefore, sentence error tendency section 13 reads the learning data stored in the spoken language resource storage unit 21, from the speech recognition result data D2 indicating the learning data, the same input speech x _m obtained by speech recognition The hypothesis w _{m, n} and other sentence hypotheses w _{m, k} are acquired, and the edit distance R (w _{m, n} , w _{m, k} ) is calculated. Note that these edit distances are treated as constants when learning the error correction model.

(Step S33: Feature Function Update Process)
The error tendency learning unit 13 updates the value of the feature function γ _j (•, •) based on the linguistic features between utterances. This is because the feature function γ _j depends on the posterior probability p (w _{m−1, n} ) of the sentence hypothesis w _{m−1, n} of the immediately preceding utterance, as is apparent from the equation (9). The posterior probability p (w _{m−1, n} ) is calculated using the equation (10) based on the current model parameters Λ and Φ. That is, the error tendency learning unit 13 obtains x _m , w, and G _m−1 in Expression (10) from the speech input x _m−1 and the speech input x _{m−1 of the} immediately preceding utterance indicated by the learning data, respectively. sentence hypotheses _{w m-1 was, n,} is calculated from the voice input _{x m-2} as a set _{G m-2} of word strings obtained as the speech recognition result. At this time, the error tendency learning unit 13 stores the score f _am (x _m−1 | w _{m−1, n} ) of the acoustic model of the sentence hypothesis w _{m−1, n in} the acoustic model storage unit 22. It is obtained using an acoustic model and input speech x _m−1 which is speech data indicated by the (m−1) th learning data. Further, the error tendency learning unit 13 obtains the language model score f _am (w _{m−1, n} ) of the sentence hypothesis w _{m−1, n} using the language model stored in the language model storage unit 23. . The error tendency learning unit 13 obtains the feature function γ _j from the feature function data D3.

(Step S34: Objective function calculation process)
The error tendency learning unit 13 calculates the value of the objective function L (Λ, Φ) according to the equation (12). The error tendency learning unit 13 calculates Χ (w _{m, n} ) in the equation (12) by the equation (13). For this calculation, the edit distance R (w _{m, n} , w _m obtained in step S32 is used. _{, K} ) and the posterior probability P (w _{m, k} | x _m ) calculated by the equation (14). The error tendency learning unit 13 uses g ^ (w _{m, k} | x _m ) used for the calculation of Expression (14), the acoustic model score f _am (x _m | w _{m, k} ), and the language model score f _lm. (W _{m, k} ) and the current model parameters Λ and Φ are calculated using equation (15). Further, the error tendency learning unit 13 uses the conditional probability P (w _{m, n} | x _m , G _m−1 ) in the equation (12) as the acoustic model score f _am (x _m | w _{m, n} ), The language model score f _lm (w _{m, n} ) and the current model parameters Λ and Φ are calculated using equation (10).
When each sentence hypothesis w _{m, n} , w _{m, k} is a sentence hypothesis w, the error tendency learning unit 13 stores the acoustic model score f _am (x _m | w) in the acoustic model storage unit 22. an acoustic model are obtained using the input and speech x _m is voice data indicated by the m-th training data. Further, the error tendency learning unit 13 obtains the score f _lm (w) of the language model of the sentence hypothesis w using the language model stored in the language model storage unit 23. Further, the error tendency learning unit 13 calculates the value of the feature function f _i (w) from the sentence hypothesis w, and uses the value calculated in step S33 as the value of the feature function γ _i (w, G _m-1 ). The error tendency learning unit 13 obtains the feature function f _i from the feature function data D3.

(Step S35: gradient calculation process)
The error tendency learning unit 13 obtains the gradients ΔΛ and ΔΦ related to the model parameters Λ and Φ in Expression (12) using Expressions (16) and (17) using the current model parameters Λ and Φ. The error tendency learning unit 13 uses Χ (w _{m, n} ) and posterior probabilities P (w _{m, n} | x _m , G _m−1 ) and P (w _{m, n} ) in the equations (16) and (17). _' | X _m , G _m-1 ) uses the value obtained when the objective function L (Λ, Φ) is calculated in step S34. In addition, the error tendency learning unit 13 calculates the feature functions γ _j (w _{m, n} , G _m−1 ) and γ _j (w _{m, n ′} , G _m−1 ) in Expression (16) in step S33. Use the value obtained. Further, the error tendency learning unit 13 uses the values calculated in step S34 for the feature functions f _i (w _{m, n} ) and f _i (w _{m, n ′} ) in the equation (17).

The error tendency learning unit 13 updates the model parameters Λ and Φ according to the equations (18) and (19) using the obtained gradients ΔΛ and ΔΦ. Note that predetermined values are used as the coefficients η _Λ and η _Φ in the equations (18) and (19).

(Step S36: End determination process)
The error tendency learning unit 13 compares the value of the objective function obtained by the gradient calculation process in step S35 with the value of the objective function before update, and if the change in value is equal to or greater than a predetermined value, the process from step S33 is performed. If it is smaller than the predetermined value, it is considered that the update has converged, the update of the model parameters Λ and Φ is aborted, and the process of step S37 is executed.

(Step S37: Error correction model output process)
The error tendency learning unit 13 stores an error correction model using the model parameters Λ = (λ ₁ , λ ₂ ,...) And Φ = (φ ₁ , φ ₂ ,. Write to part 24.

[4.4 Step S4]
When the input speech data D4 is input as speech recognition target speech data, the speech recognition unit 14 stores the error correction model stored in the error correction model storage unit 24 and the acoustic stored in the acoustic model storage unit 22. Using the model and the language model stored in the language model storage unit 23, a word string of correct answer candidates of the input speech data D4 is obtained, and their scores are calculated. The voice recognition unit 14 outputs the voice recognition result data D5 in which the correct candidate word string having the best score is set as the correct word string in real time. By using this error correction model, the speech recognition unit 14 corrects an error in the selection of the speech recognition result obtained from the input speech data D4.

[5. effect]
According to the present embodiment, the speech recognition apparatus 1 can configure an error correction model reflecting the content of the immediately preceding utterance without a correct word string, and recognition errors can be reduced compared to conventional speech recognition.

[6. Others]
The voice recognition device 1 described above has a computer system inside. The operation process of the speech recognition apparatus 1 is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

DESCRIPTION OF SYMBOLS 1 Speech recognition apparatus 11 Speech recognition part 12 Feature-value extraction part 13 Error tendency learning part 14 Speech recognition part 21 Spoken language resource storage part 22 Acoustic model storage part 23 Language model storage part 24 Error correction model storage part

Claims (7)

A speech language resource storage unit that stores speech recognition results obtained by speech recognition of speech data of speech, while retaining the order of speech;
Acquire linguistic features according to the order of utterances from words included in the speech recognition results and words included in the speech recognition results of utterances earlier than the speech recognition results, and the acquired linguistic features Are weighted according to the posterior probabilities of the speech recognition results of the past utterances, the tendency of recognition errors of words is statistically learned based on the weighted linguistic features, and the tendency of the learned recognition errors is An error tendency learning unit that generates an error correction model for correction;
A speech recognition apparatus comprising: The error tendency learning unit statistically calculates a tendency of recognition error of a word based on a linguistic feature corresponding to the weighted utterance order and a linguistic feature in the same utterance obtained from the speech recognition result. And generate an error correction model to correct the tendency of learned recognition errors.
The speech recognition apparatus according to claim 1. The linguistic feature according to the utterance order is a co-occurrence relationship between a word included in the speech recognition result and a word included in the speech recognition result of the past utterance,
The linguistic features in the same utterance are a co-occurrence relationship of a plurality of consecutive words in the same utterance obtained from the speech recognition result, a co-occurrence relationship of a plurality of non-consecutive words, syntactic information of words, or One or more of the semantic information of the word,
The speech recognition apparatus according to claim 2. The error correction model includes a first feature function based on a linguistic feature according to the order of the utterances, weighted by a posterior probability of the speech recognition result of the past utterance, and a linguistic feature in the same utterance. A calculation formula for correcting a score of speech recognition using a second feature function based on a unique feature and feature weights of each of the first feature function and the second feature function,
The error tendency learning unit includes a value of the first feature function obtained from the speech recognition result and the speech recognition result of the past utterance, and the second feature function obtained from the speech recognition result. Calculated by an evaluation function determined using a value and a value obtained by weighting the number of word errors obtained by comparing a plurality of the speech recognition results obtained from the same speech data with the posterior probability of the speech recognition results Statistically calculating the feature weight based on the evaluated value, and generating the error correction model using the calculated feature weight;
The voice recognition apparatus according to claim 2 or claim 3, wherein A speech recognition unit that recognizes input speech data and corrects an error in selecting a speech recognition result obtained from the input speech data using the error correction model generated by the error tendency learning unit Further comprising
The voice recognition device according to claim 1, wherein the voice recognition device is a voice recognition device. Spoken language resource storage process for storing speech recognition results obtained by speech recognition of speech data, while maintaining the order of speech,
Acquire linguistic features according to the order of utterances from words included in the speech recognition results and words included in the speech recognition results of utterances earlier than the speech recognition results, and the acquired linguistic features Are weighted according to the posterior probabilities of the speech recognition results of the past utterances, the tendency of recognition errors of words is statistically learned based on the weighted linguistic features, and the tendency of the learned recognition errors is An error tendency learning process to generate an error correction model for correction;
An error correction model learning method characterized by comprising: Computer
A speech language resource storage means for storing speech recognition results obtained by speech recognition of speech data of speech, while retaining the order of speech;
Acquire linguistic features according to the order of utterances from words included in the speech recognition results and words included in the speech recognition results of utterances earlier than the speech recognition results, and the acquired linguistic features Are weighted according to the posterior probabilities of the speech recognition results of the past utterances, the tendency of recognition errors of words is statistically learned based on the weighted linguistic features, and the tendency of the learned recognition errors is An error tendency learning means for generating an error correction model for correction;
A program for causing a voice recognition apparatus to function.

Images