JP6300394B2 - Error correction model learning device and program - Google Patents

Description

  The present invention relates to an error correction model learning device and a program.

  For error correction in speech recognition, statistical error correction models obtained as a result of learning by statistically learning the tendency of speech recognition errors using linguistic features from speech and transcriptions (correct sentences) There is a technology for improving the performance of speech recognition by using (see, for example, Non-Patent Document 1). In this prior art, model parameters of an error correction model are learned using speech data, the speech recognition result and a correct word string.

Kobayashi et al., “News speech recognition by discriminative scoring based on word error minimization”, IEICE Journal, vol.J93-D no.5, 2010, p. 598-609

In an application system such as a caption production system, it is necessary to process input audio sequentially and in real time. However, the learning of the error correction model according to the prior art has a large amount of calculation, so that there is a problem in using the above application from the viewpoint of sequentiality and real-time property.
For example, in the prior art, it is necessary to align input speech in order to obtain the utterance time of each word constituting a correct word string. At the time of alignment, the acoustic score (score based on log likelihood based on a statistical acoustic model) is calculated in accordance with the utterance time of each word. This is because an acoustic score is required in the error correction model of the prior art. Although the alignment process is important for accurate model parameter estimation in error correction model learning, it is not preferable from the viewpoint of computational complexity. Even if the error correction model can be learned sequentially, the application of the model will be delayed, so it is considered that the effectiveness of the error correction model will be greatly impaired in tasks where the topic changes rapidly, such as broadcast audio. It is done.

  The present invention has been made in view of such circumstances, and provides an error correction model learning apparatus and program that can easily learn error correction models sequentially.

[1] According to one aspect of the present invention, a speech recognition result correcting unit that corrects a speech recognition result according to an input instruction, a linguistic feature included in the speech recognition result, and a correction result by the speech recognition result correcting unit An error that learns an error tendency of a word from a difference from a linguistic feature included in an error and updates an error correction model for correcting the error tendency of the word in speech recognition according to the learned error tendency of the word An error correction model learning device comprising: a correction model update unit.
According to the present invention, the error correction model learning device learns an error tendency of a word based on a difference in linguistic features included in each of a speech recognition result and a correction result obtained by manually correcting the speech recognition result. The error correction model is updated according to the learned error tendency.
As a result, the error correction model learning device learns the error tendency of the word with a small amount of calculation using only the speech recognition result of the sequentially input speech and the correction result obtained by manually correcting the speech recognition result. The modified model can be updated. Therefore, the error correction model learning device can update the error correction model sequentially and with low delay.
Note that the “input instruction” may refer to manual correction work.

[2] One aspect of the present invention is the error correction model learning device described above, wherein the error correction model update unit includes the frequency of co-occurrence of words or parts of speech included in the speech recognition result, and the correction. The tendency of word errors is learned using the frequency of words or part-of-speech co-occurrence included in the results.
According to this invention, the error correction model learning device uses the frequency of co-occurrence of words or word parts of speech included in each of the speech recognition result and the correction result obtained by manually correcting the speech recognition result. Learn trends and update error correction models.
Thereby, the error correction model learning device efficiently learns the error tendency of words in speech recognition by counting the frequency of co-occurrence of words or parts of speech of speech recognition results and correction results, and from the learning results. The error correction model can be updated.

[3] One aspect of the present invention is the error correction model learning apparatus described above, wherein the linguistic feature is a frequency of a continuous word string or a continuous word part-of-speech string, and the error correction model. Is a calculation formula for correcting a score of speech recognition using a feature function based on the linguistic feature and a feature weight of the feature function, and the error correction model update unit is configured to calculate the feature of the error correction model. The weight is updated according to the tendency of the learned error of the word.
According to the present invention, the error correction model learning device can detect a word from a difference in linguistic features such as the frequency of consecutive word strings and the frequency of part of speech strings of consecutive words included in each of the speech recognition result and the correction result. Learn error tendency. An error correction model is a calculation formula that corrects a speech recognition score based on a feature function that represents a linguistic feature and a feature weight of the feature function. The error correction model learning device performs an error according to the tendency of a learned word error. Update the feature weight of the modified model.
Thereby, the error correction model learning device can efficiently learn the recognition error tendency of speech recognition and can update the feature weight in the error correction model.

[4] One aspect of the present invention is the error correction model learning apparatus described above, which recognizes an input speech and uses the error correction model updated by the error correction model update unit, from the input speech. It further comprises a speech recognition unit that corrects and outputs an error in selection of the obtained speech recognition result.
According to the present invention, the error correction model learning device selects a speech recognition result using an error correction model that is sequentially updated from among correct answer candidates obtained by speech recognition of input speech.
Thereby, even when a topic changes, the error correction model learning device can obtain a speech recognition result with a high recognition rate by using an error correction model that is sequential and has a small delay time due to learning according to the topic. it can.

[5] According to one aspect of the present invention, a computer recognizes a speech recognition result correcting unit that corrects a speech recognition result according to an input instruction, a linguistic feature included in the speech recognition result, and the speech recognition result correcting unit. Learns the error tendency of the word from the difference from the linguistic feature included in the correction result by, and an error correction model for correcting the error tendency of the word in speech recognition according to the learned error tendency of the word An error correction model update unit for updating is a program for causing an error correction model learning device to function.

  According to the present invention, an error correction model can be sequentially learned easily.

It is a figure which shows the correction process of the speech recognition result applied to one Embodiment of this invention. It is a figure which shows the sequential update process of the error correction model by the embodiment. It is a figure which shows the update process of the language model by the embodiment. It is a functional block diagram which shows the structure of the error correction model learning apparatus by the embodiment. It is a flowchart which shows the whole process of the error correction model learning apparatus by the embodiment. It is a flowchart which shows the process of the morphological analysis part by the embodiment. It is a flowchart which shows the process of the error correction model update part by the embodiment. It is a flowchart which shows the process of the language model update part by the embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[1. Overview of this embodiment]
An error correction model based on discriminative learning that reflects the error tendency of speech recognition has already been devised. In conventional error correction model learning, a procedure for aligning the utterance start time and end time of each word constituting a word string is necessary for a given voice and a correct word string corresponding to the voice. . The procedure is as follows.

(1) Align voice data and correct word strings. At this time, an acoustic score and a language score (score based on a word string generation probability based on a statistical language model) of each word constituting the correct word string are calculated.
(2) The expected number of errors (average number of word errors in speech recognition) for the input speech data is calculated on the basis of the speech recognition result and the correct word strings already aligned in (1).
(3) The model parameters of the error correction model are estimated using the expected number of errors as an evaluation function.

  In the prior art described above, the model parameters of the error correction model are estimated with all audio data given. If the above procedure is repeated every time voice data is obtained, the conventional technique can be applied to sequential processing, but the processes (1) and (2) are calculations for parameter estimation of an error correction model. It is a factor that pushes up time. Therefore, it is not suitable for an application that requires real-time performance, such as applying the estimated model parameter to immediate speech recognition. Therefore, it is desirable to learn a model easily without performing an alignment procedure.

  The error correction model learning apparatus according to the present embodiment uses an error correction model applied to a speech recognition application in which a correct word string in which an input speech recognition error is corrected is given by a caption production system or the like as a correct word string. Learning sequentially and simply from the speech recognition results and the correction results without considering the alignment procedure.

[2. Error correction model learning algorithm]
The error correction model learning apparatus according to the present embodiment learns an error correction model based on sequential processing with excellent real-time characteristics in order to solve the problem of the conventional method. In the learning algorithm of this embodiment, an error correction model applied to an application in which correct word strings are sequentially obtained as in a caption production system is targeted for learning.

  FIG. 1 is a diagram illustrating a correction process for obtaining a correction result from a voice recognition result. In a general correction process, a speech recognition result is sequentially given by the speech recognition device at the previous stage. In the case of a conventional sequential confirmation type speech recognition apparatus (for example, see Japanese Patent Application Laid-Open No. 2001-092496), the recognition result is not sent in a word corresponding to the input utterance, but in the speech recognition apparatus. A partial word string that has been confirmed (determined to be the most correct answer) is sent. This process is also applied in this embodiment.

Thereafter, the speech recognition result is corrected by manual work, and the correction result is sent to the subsequent processing as a correct word string. In the correction process, normally, the corrected word string is not sequentially sent to the subsequent stage, but all the corresponding words of the input utterance are sent to the subsequent stage. Accordingly, one unit of correction (correction block) in the correction process is a block with utterance as a unit.
For example, in the case of a caption production system, the subsequent process refers to a process of converting a correct word string into a caption format and superimposing it on a broadcast wave.

The correction work refers to the following three works for each word of the speech recognition result.
(1) Delete words included in the speech recognition result. This corresponds to correction of a recognition error called “insertion error”.
(2) Replace words included in the speech recognition result. This corresponds to correction of a recognition error called “replacement error”.
(3) Insert a word that is not in the speech recognition result. This corresponds to correction of a recognition error called “deletion error”.

  FIG. 2 is a diagram showing a sequential update process of the error correction model according to the present embodiment. In this embodiment, the error correction model is sequentially learned and applied to speech recognition. Therefore, as shown in the figure, multiple correction blocks are defined as one sequential update block, and the update timing of the error correction model is adjusted. To do. Then, the error correction model is sequentially updated using only the speech recognition result and the word string of the correction result included in the sequential update block.

[2.1 Error correction model of conventional method]
According to Bayes' theorem, when speech input x is given, the most likely word sequence w ^ (“^” represents “hat”) for this speech input x is expressed by the following equation (1). ).

The voice input x and the word string w correspond to, for example, the unit of speech, and P (w | x) is a posterior probability that a word string (sentence hypothesis) w is obtained when the voice input x is given.
P (x | w) is a likelihood indicating acoustic likelihood for the word string w, and the score (acoustic score) is a hidden Markov model (HMM) and a mixed Gaussian distribution (Gaussian Mixture). It is calculated based on a statistical acoustic model (hereinafter referred to as “acoustic model”) typified by Model, GMM). In other words, when an acoustic feature amount is given, a score representing the likelihood of each of a plurality of correct candidate words is an acoustic score.

On the other hand, P (w) is a linguistic generation probability for the word string w, and the score (language score) is described as a statistical language model (hereinafter, “language model”) such as a word n-gram model. ). In other words, when a word string before or after a speech recognition target word, or both word strings before and after the given word string, a score representing the likelihood of each of a plurality of correct answer word strings is a language score. The word n-gram model gives the occurrence probability of the next word from the history of the word (N-1) based on the statistics of N word chains (N is 1, 2, or 3, for example). It is a model.
In the following description, HMM-GMM is used for the acoustic model and n-gram is used for the language model.

  When P (x | w) P (w) in Equation (1) is maximum, the logarithm is also maximum. Therefore, in speech recognition, the evaluation function S (w | x) of the word string w, which is a sentence hypothesis (correct answer candidate) when the speech input x is given, based on the Bayes' theorem of the above equation (1). It is defined as the following formula (2).

In formula (2), f _am (x | w) is a logarithmic acoustic score of the word sequence w according to the acoustic model, f _lm (w) is a logarithmic language score of the word sequence w according to the language model, and λ _lm is an acoustic score Is the weight of the language score for.

  When the formula (2) is determined, as shown in the following formula (3), the evaluation function S (w | x shown by the formula (2) is selected from the set of correct candidate word strings w for the speech input x. ) Is selected as the speech recognition result of speech input x.

  In the error correction model in the conventional method such as Non-Patent Document 1, the evaluation function of the hypothesis (correct candidate word string w) is set as the following expression (4), and the word string w ^ that is the maximum likelihood hypothesis is expressed as the following expression (5 )

The right side of Equation (4) is an error correction model. Equation (4) in _{Σ i λ i f i (w} ) is a score reflecting the error tendency of the word sequence w, act as a penalty or reward for the word sequence w. Also, f _i (w) (i = 1,...) Is the i-th feature function, and the element λ _i of the model parameter Λ = {λ ₁ ,...} Is the feature function f _i (w). Weight (feature weight). The feature function is defined as a function that becomes the number if a linguistic rule is established in a given word string (here, word string w), and is 0 if not established. These rules are, for example, linguistic features such as consecutive words in the same utterance, co-occurrence relationship of two or more words that are not consecutive, syntactic information or semantic information of words. Examples of rules specific feature function f _i in the conventional method, and the like below.

  For example, there are the following (1) and (2) as feature functions based on the co-occurrence relationship of words.

(1) A function that returns the number of consecutive words (u, v) when the word string w includes a continuous word pair (u, v) (2) A case where the word string w includes a non-continuous word pair (u, v) , A function that returns the number

  For example, the following (3) and (4) are feature functions based on syntax information obtained by replacing each word constituting the word string w with a part-of-speech category (syntax information) such as a noun or a verb. Note that c (•) is a function that maps words to parts of speech.

(3) A function that returns the number of pairs of parts of speech (c (u), c (v)) that are consecutive in the word string w (4) A pair of parts of speech that are not consecutive in the word string w (c ( u), c (v)), a function that returns the number if it is included

  Alternatively, for example, the following (5) and (6) are feature functions based on semantic information obtained by replacing each word constituting the word string w with a category (semantic category) representing semantic information. is there. The semantic category can be obtained by using a thesaurus stored in a database provided externally or internally in the error correction model learning apparatus of the present embodiment. Note that s (•) is a function that maps words to semantic categories.

(5) A function that returns the number of consecutive semantic categories (s (u), s (v)) in the word string w (6) A pair of semantic categories that are not consecutive in the word string w ( a function that returns the number of s (u), s (v))

In the method of Non-Patent Document 1, the calculation of the hypothesis score in Equation (4) is performed for the corrected correct word string. However, since this calculation takes time, the real-time property is lost due to the application of the error correction model. It will be broken.
Therefore, the following algorithm is used as a method for updating the model parameter Λ of the error correction model in the sequential update processing block.

[2.2 Learning algorithm of error correction model applied to this embodiment]
Now, let w be the speech recognition result in the sequential update processing block of interest, and let w ^{ref be the} correct word string that is the correction result. The score of the speech recognition result w calculated by the equation (4) using the model parameter Λ is S ^ (w | x; Λ), and the score of the correct word string w ^ref is S ^ (w ^ref | x; Λ). Then, consider the difference L ₁ (Λ) of the scores shown in the following equation (6).

If the score for the correct word string w ^ref is large, the probability that the correct word string will be selected in equation (6) increases. On the other hand, if the score for the speech recognition result w containing an error is small, the probability that such a word string is selected becomes small. Therefore, the larger L ₁ (Λ) is, the higher the probability that the correct word string will be selected.
That is, by estimating the model parameter Λ so as to increase L ₁ (Λ), an error correction model reflecting the error tendency of the speech recognition result can be obtained. Therefore, when the gradient related to the weight λ _i is obtained in order to maximize L ₁ (Λ), the following equation (7) is obtained.

Assuming that n _m correction blocks are included in the sequential update block b _m (m = 1,...), The score difference L ₁ (Λ) in the sequential update block b _m is expressed by the following equation: It is rewritten as (8). The input speech _{x n} modified corresponding to the block n (n = 1, ..., n m) the speech recognition result in the _{w n,} the correct word sequence which is a modification result as _w ^{n ref.}

Thus, the gradient [Delta] [lambda] _i ^m of the function shown in equation (8) in the sequential update block _{b m,} and becomes the following equation (9).

The value of the feature function f _i is the number of linguistic rules that occur in the speech recognition result and the correct word string based on the definition. Accordingly, the score difference L ₁ (Λ) can be maximized by a simple process of counting up the features in the sequential processing update block without calculating the score according to the formula (4) for the corrected correct word string. As a result, an error correction model can be learned from the model parameter Λ at that time.

If it is assumed that the model parameter Λ is estimated by iterative updating for the sequential update block, the weight λ _i ^m−1 is obtained in the m−1th sequential update block b _m−1 , and the weight in the current block m. λ _i ^m is expressed by the following equation (10).

In equation (10), η is a predetermined coefficient.
Alternatively, the following equation (11) can be obtained by performing weighted addition on K sequential update blocks obtained in the past.

In equation (11), ρ _k is a predetermined weight, and Σρ _k = 1 (k = 0,..., K−1).

On the other hand, in speech recognition, instead of uniquely obtaining the maximum likelihood sequence as a speech recognition result, L sequences that are a plurality of correct candidate candidates are often generated simultaneously when the maximum likelihood sequence is derived. Now, the score of the correct candidate sentence w _n ^l calculated by Expression (4) is w _n ^l for the l-th (l = 1,...) Correct answer sentence (sequence that is a correct candidate) for the speech x _n . Is S (w _n ^l | x; Λ), the posterior probability p (w _n ^l | x _n ) that the correct candidate sentence w _n ^l is generated is given by the following equation (12). It should be noted that the correct candidate sentence _{w n} ^{l 'is} the correct candidate sentence other than the correct candidate sentence _w ^{n l} of voice _{x n.}

Here, the difference L ₂ (Λ) between the score of the correct word string w _n ^ref of the speech x _n and the average score of the L correct candidate sentences w _n ^l is determined as in the following equation (13).

  When the gradient is obtained in order to maximize the function shown in Expression (13), the following Expression (14) is obtained.

Thus, sequential gradient [Delta] [lambda] _i ^m in the update block _{b m} is given by the following expression (15).

  The final update formula of the model parameter Λ is the above-described formula (10) or formula (11).

[3. Configuration of error correction model learning device]
FIG. 4 is a functional block diagram showing the configuration of the error correction model learning device 1 according to an embodiment of the present invention, and only functional blocks related to the present embodiment are extracted and shown. The error correction model learning device 1 is realized by a computer device, and, as shown in the figure, a speech recognition unit 2, a pronunciation dictionary storage unit 3, a language model storage unit 4, an acoustic model storage unit 5, and an error correction model storage unit 6 A speech recognition result correction unit 7, a morpheme analysis unit 8, a morpheme analysis dictionary database (DB) storage unit 9, an error correction model update unit 10, a language model update unit 12, and a pronunciation dictionary database (DB) storage unit 14. Composed.

  The pronunciation dictionary storage unit 3 stores a pronunciation dictionary indicating pairs of words and pronunciations. The language model storage unit 4 stores a language model. The acoustic model storage unit 5 stores an acoustic model. The error correction model storage unit 6 stores an error correction model. As the speech recognition unit 2, a conventional speech recognition device can be used. The pronunciation dictionary stored in the pronunciation dictionary storage unit 3, the language model stored in the language model storage unit 4, and the acoustic model storage unit 5 And the error recognition model stored in the error correction model storage unit 6 are used to obtain the speech recognition result of the input speech D1 that is sequentially input. Since the error correction model stored in the error correction model storage unit 6 is sequentially updated by the error correction model update unit 10, the speech recognition unit 2 uses the updated error correction model for speech recognition. In addition, since the pronunciation model stored in the pronunciation dictionary storage unit 3 and the language model stored in the language model storage unit 4 are updated by the language model update unit 12, the speech recognition unit 2 performs the updated pronunciation. Dictionary and language model are used for speech recognition. The voice recognition unit 2 outputs the voice recognition result to the voice recognition result correction unit 7.

  The speech recognition result correction unit 7 manually corrects the speech recognition result to a correct word string, and outputs the speech recognition result and the correct word string that is the correction result to the morpheme analysis unit 8. For the speech recognition result correction unit 7, for example, a conventional caption production system can be used. The morpheme analysis unit 8 performs morpheme analysis on the corrected character string included in the correct word string. The morpheme analysis unit 8 outputs the speech recognition result and the correct word string and the morpheme analysis result to the error correction model update unit 10. The error correction model update unit 10 uses the speech recognition result obtained by the speech recognition unit 2 and the correct word string obtained as a result of the morphological analysis by the morpheme analysis unit 8 after correction by the speech recognition result correction unit 7. Estimate the model parameter Λ of the modified model. The error correction model update unit 10 sequentially estimates the model parameter Λ every time the speech recognition result for the sequentially updated block and the correct word string of the correction result are obtained, and stores them in the error correction model storage unit 6 using the estimated model parameter Λ. Sequentially update the error correction model. The storage unit 11 included in the error correction model update unit 10 stores a speech recognition result used for updating the error correction model and a correct word string of the correction result.

  The pronunciation dictionary database storage unit 14 stores a pronunciation dictionary database. The pronunciation dictionary stored in the pronunciation dictionary storage unit 3 is a part of the pronunciation dictionary database stored in the pronunciation dictionary database storage unit 14 that is used for speech recognition. The language model update unit 12 updates the language model stored in the language model storage unit 4 based on the correct word string. Furthermore, if the pronunciation of the word included in the correct word string is not registered in the pronunciation dictionary stored in the pronunciation dictionary storage unit 3, the language model update unit 12 is stored in the pronunciation dictionary database storage unit 14. Read and register the pronunciation of the word from the pronunciation dictionary database. The storage unit 13 included in the language model update unit 12 stores a correct word string used for updating the language model.

[4. Processing procedure of error correction model learning device]
FIG. 5 is a flowchart showing the overall processing of the error correction model learning device 1. The error correction model learning device 1 sequentially performs the processing shown in FIG. Hereinafter, processing of each step shown in FIG.

[4.1 Step S1]
The speech recognition unit 2 includes a pronunciation dictionary stored in the phonetic dictionary storage unit 3, a language model stored in the language model storage unit 4, an acoustic model stored in the acoustic model storage unit 5, and an error correction model. Using the error correction model stored in the storage unit 6, the input speech D1 is recognized as speech. This speech recognition result is obtained by correcting an error in selection of a speech recognition result obtained from input speech by an error correction model. The voice recognition unit 2 outputs a word string that is a voice recognition result of the input voice. In the present embodiment, the speech recognition unit 2 sequentially recognizes the input speech and performs speech recognition result data D2 indicating a confirmed recognition result in units of words, as in the technique disclosed in Japanese Patent Laid-Open No. 2001-092496. It outputs to the speech recognition result correction part 7 of a back | latter stage one after another. Further, the speech recognition unit 2 outputs utterance end symbol data indicating the utterance end symbol to the speech recognition result correction unit 7 when the input speech reaches the end point of the utterance (the utterance boundary defined by the silent section). The utterance end symbol is necessary for the speech recognition result correction unit 7 to determine the boundary of the correction block. In the present embodiment, the speech recognition result indicated by the speech recognition result data D2 may be only a word string of the maximum likelihood sequence that is sequentially determined, or a plurality of correct candidate word sequences (a word sequence of the maximum likelihood sequence and one or more other word sequences). Correct candidate word string). When the speech recognition result is composed of a plurality of correct candidate word strings, the speech recognition score of each correct candidate word string is also added to the speech recognition result data D2.

[4.2 Step S2]
When the speech recognition result data D2 is sent from the preceding speech recognition unit 2, the speech recognition result correction unit 7 indicates an error of each word constituting the speech recognition result indicated by the speech recognition result data D2 as input means (not shown) The correction result corrected according to the instruction input by the person is generated. However, when the speech recognition result is composed of a plurality of correct candidate word strings, the correction target is the maximum likelihood sequence. The words constituting the recognition result are sequentially transmitted from the speech recognition unit 2 by the speech recognition result data D2, and the unit (correction block) for performing manual correction is from the beginning to the end of the utterance. The speech recognition result correction unit 7 determines the boundary of the correction block based on the utterance end symbol data output from the speech recognition unit 2.
When a recognition operation of each word in the correction block is corrected manually and then a sending operation is performed by an input unit (not shown), the speech recognition result correction unit 7 sends the speech recognition result data D2 and the morphological analysis unit 8 in the subsequent stage. A pair with correction result data D3 indicating the correction result is output in units of correction blocks.

[4.3 Step S3]
The morpheme analysis unit 8 performs morpheme analysis on the correction result word string indicated by the correction result data D3 input from the speech recognition result correction unit 7 in the previous stage. Although the speech recognition result is a word string, in the correction work, character input is performed without considering the division of words, so the word boundary of the portion where the correction has been performed is unknown. Therefore, the morphological analysis unit 8 sequentially checks words included in the correction result, and decomposes words that are not divided into words into words by morphological analysis.
FIG. 6 is a flowchart showing the processing of the morphological analysis unit 8.

(Step S31: Correction Result Word Selection Process)
The morpheme analysis unit 8 compares the correction result indicated by the correction result data D3 with the maximum likelihood sequence indicated by the speech recognition result data D2, and specifies a mismatched portion as a correction character string. The morphological analysis unit 8 selects the first (m = 1) corrected character string that has not yet been processed from the identified m = 1,..., Corrected character string.

(Step S32: Unregistered word determination process)
The morpheme analysis unit 8 determines whether or not the m-th corrected character string is composed of a plurality of words, and the corrected character string is included in the morpheme analysis dictionary database storage unit 9. Judgment by whether or not. If the modified character string is included in the morphological analysis dictionary, the morpheme analysis unit 8 determines that the mth modified character string is one word, and performs the process of step S34. On the other hand, when the mth modified character string is not included in the morpheme analysis dictionary, the morpheme analysis unit 8 considers that the modified character string is composed of a plurality of word strings, and performs the process of step S33.

(Step S33: Morphological analysis process)
The morpheme analysis unit 8 performs a morphological analysis on the corrected character string and searches for a combination of known words included in the morpheme analysis dictionary. For example, if the corrected character string c is the k-th word as a correction result, the word string w ^ obtained from the corrected character string c by morphological analysis can be obtained by the following equation (16).

Here, the word string w _k−1 and the word string w _{k + 1} are words before and after the corrected character string c in the correction result, respectively. This is the same technique as the conventional technique found in statistical morphological analysis. The morpheme analysis unit 8 replaces the m-th corrected character string c in the correction result with the word string w ^ obtained as a result of the morphological analysis.

(Step S34: Next Correction Word Selection Process)
The morpheme analyzer 8 adds 1 to the current value of m, and repeats the processing from step S32. And the morpheme analysis part 8 will complete | finish the processing flow of FIG. 6, if the process of step S32-S33 is performed about all the correction word strings contained in a correction result. The morpheme analysis unit 8 includes the speech recognition result data D2 and the correction result data D4 indicating the correction result obtained by replacing the correction character string included in the correction result indicated by the correction result data D3 with the word string obtained as a result of the morphological analysis The data is output to the error correction model update unit 10 at the subsequent stage.

[4.4 Step S4]
When the error correction model update unit 10 receives the speech recognition result data D2 from the preceding morpheme analysis unit 8 and the correction result data D4 indicating the correction result on which the morphological analysis has been performed as input, the error correction model update unit 10 receives the speech recognition result data D2 and the correction result. Data D4 is written in the storage unit 11 provided inside. The error correction model updating unit 10 updates the model parameter Λ based on the speech recognition result indicated by the speech recognition result data D2 stored in the storage unit 11 and the correction result indicated by the correction result data D4, and the updated model parameter The error correction model stored in the error correction model storage unit 6 is updated by Λ.
FIG. 7 is a flowchart showing the processing of the error correction model update unit 10.

(Step S41: Update determination process)
The error correction model update unit 10 determines whether to update the model parameter Λ of the error correction model. This determination is performed in order to adjust the update frequency of the model. The error correction model update unit 10 assumes that the number of words in the speech recognition result that has not yet been used in the subsequent processing is N, and the number of words in the correction result is M. If the smaller of the number M is equal to or greater than a predetermined threshold value, it is determined that the subsequent process is performed. The error correction model update unit 10 stores the number of words N of the speech recognition result and the number of words M of the correction result in the storage unit 11 and has not yet been used for subsequent processing, the speech recognition result data D2, and the correction result Obtained from data D4. However, when the speech recognition result is composed of a plurality of correct answer candidate word strings, the number N of words in the speech recognition result is set as the number of words included in the maximum likelihood sequence.

When the smaller number of words N of the speech recognition result or the number of words M of the correction result is smaller than the threshold value, the error correction model update unit 10 determines not to perform the subsequent process, and ends the process of FIG. To do. The speech recognition result data D2 and the correction result data D4 written in the storage unit 11 are used for the next update opportunity.
On the other hand, if the smaller one of the number of words N of the speech recognition result or the number of words M of the correction result is equal to or greater than the threshold, the error correction model update unit 10 determines to perform the processing from step S42. The speech recognition result data D2 and the correction result data D4 that are stored in the storage unit 11 and have not been used for the processing after step S42 are set as the sequential update block b _m .

(Step S42: feature calculation process)
The error correction model update unit 10 performs a feature calculation process on the sequential update block b _m . Specifically, the error correction model update unit 10 sets the following feature function values (for each of the speech recognition result indicated by the speech recognition result data D2 of the sequential update block b _m and the correction result indicated by the correction result data D4) ( Number of features).

(1) When a continuous word pair (u, v) is included, a function that returns the number (2) When a continuous part-of-speech pair (c (u), c (v)) is included, A function that returns a number (3) A function that returns a number when a series of consecutive semantic categories (s (u), s (v)) is included

As a result, f _i (w _n ^ref ) is obtained from the correct word string w _n ^ref indicated by the correction result. Also, if the speech recognition result of the maximum likelihood sequence, the speech recognition result of the maximum likelihood sequence w _n from f _{i (w n)} is obtained, the speech recognition result is more correct candidate word sequence (correct candidate sentence) w _n ^l In this case, f _i (w _n ^l ) is obtained.

(Step S43: gradient calculation process)
The error correction model update unit 10 performs gradient calculation processing for calculating a gradient value (difference between feature function values) according to the equation (9) or the equation (15) using the value of the feature function obtained in step S42. Do.
Specifically, when the speech recognition result indicates the maximum likelihood sequence, the error correction model update unit 10 substitutes f _i (w _n ^ref ) and f _i (w _n ) obtained in step S42 into equation (9). Then, the gradient Δλ _i ^m is calculated.
On the other hand, when the speech recognition result indicates a plurality of correct candidate word strings, the error correction model update unit 10 ^obtains the score S ^ (w _n ^l | x _n ) of each correct candidate word string w _n ^l from the speech recognition result data D2. Read-out, and a posteriori probability p (w _n ^l | x _n ) of each correct answer candidate word string w _n ^l is calculated by equation (12). Subsequently, the error correction model update unit 10 performs f _i (w _n ^ref ) and f _i (w _n ^l ) obtained in step S42 and the posterior probability p (w _n ) of each calculated correct candidate word string w _n ^{l. l} | x _n ) is substituted into equation (15) to calculate the gradient Δλ _i ^m .

(Step S44: Parameter update process)
The error correction model update unit 10 acquires the weight λ _i ^m−1 calculated for the sequential update block b _m−1 from the error correction model stored in the error correction model storage unit 6, and the gradient Δλ obtained in step S43. by using the _i ^m, read the weight λ _i ^m-1, it calculates the weight lambda _i ^m by the equation (10). The weight λ _i ^m−1 may be read from the storage unit 11.
Alternatively, error correction model update section 10, the sequential update block _{b m-1 ~b m- (K} -1) weighting ^{_{λ i m-1 ~λ i m-}} (K-1) to the storage unit 11 calculated for read out. The error correction model update unit 10 uses the gradient Δλ _i ^m obtained in step S43 and the read weights λ _i ^{m−1 to} λ _i ^{m− (K−1)} , and uses the weight λ _{i according} to the equation (11). ^m is calculated. In addition, when the error correction model used before the current error correction model is also stored in the error correction model storage unit 6, the weights λ _i ^{m−1 to} λ _i ^{m− (K−1)} are assigned to these. You may acquire from an error correction model.
The error correction model update unit 10 writes the model parameter Λ including the calculated weight λ _i ^m in the storage unit 11.

(Step S45: Model update process)
The error correction model update unit 10 is stored in the error correction model storage unit 6 according to the timing when the speech recognition unit 2 detects the end of the utterance of the input speech (before the start of the subsequent utterance), and the speech recognition unit 2 refers to it. The model parameter of the error correction model is replaced with the model parameter Λ held in the storage unit 11. The error correction model update unit 10 ends the process of FIG.

[4.5 Step S5]
The language model update unit 12 updates the language model stored in the language model storage unit 4 using the correction result. In the present embodiment, the language model is updated by sequential processing, similar to the error correction model. Moreover, the update method is based on the linear interpolation of the n-gram model which is a conventional method. However, the language model updating unit 12 updates not only the language model but also the pronunciation dictionary stored in the pronunciation dictionary storage unit 3 and referred to by the speech recognition unit 2. This is a process for enabling speech recognition of words that are not included in the pronunciation dictionary currently in use, and is necessary for improving the effect even in the error correction model.

  FIG. 3 is a diagram illustrating a language model update process. In the n-gram model, it is necessary to learn from as much text data as possible in order to guarantee the statistical accuracy of the model. Therefore, as shown in the figure, in the update process of the language model, a plurality of sequential update blocks of the error correction model are combined, and a block from which a sufficient number of words are obtained is regarded as one unit of update. A block that is a unit for updating the language model is a language model update block.

  FIG. 8 is a flowchart showing the processing of the language model update unit 12. When receiving the input of the correction result data D4 sent from the morphological analysis unit 8, the language model update unit 12 writes the correction result data D4 in the storage unit 13 provided therein.

(Step S51: Update determination process)
The language model update unit 12 determines whether to update the language model. This determination is performed for the purpose of estimating a language model after obtaining a sufficient number of words from the correction result. The language model update unit 12 determines that the subsequent process is performed when the number M of words in the correction result is equal to or greater than a predetermined threshold. The language model update unit 12 acquires the number of words M as a correction result from the correction result data D4 stored in the storage unit 13 and not yet used for the subsequent processing. If the number M of correction results is smaller than the threshold value, the language model update unit 12 determines not to perform the subsequent process, and ends the process of FIG. The correction result data D4 written in the storage unit 13 is used for the next update opportunity.
On the other hand, when the number M of corrected words is equal to or greater than the threshold, the language model update unit 12 determines that the subsequent process is to be performed, and performs the process of step S52. The language model update unit 12 sets the correction result data D4 stored in the storage unit 13 and not yet used for the processing after step S52 as a language model update block to be processed.

(Step S52: Pronunciation dictionary update processing)
The language model update unit 12 reads correction result data D4 included in the language model update block to be processed from the storage unit 13. The language model update unit 12 reads and assigns the pronunciation from the pronunciation dictionary database stored in the pronunciation dictionary database storage unit 14 to the word included in the correction result indicated by the correction result data D4. The pronunciation dictionary stored in the pronunciation dictionary storage unit 3 is referred to by the voice recognition unit 2, but what is to be updated is a set of words and pronunciations not included in the pronunciation dictionary storage unit 3. Therefore, for example, the language model updating unit 12 inquires the pronunciation dictionary storage unit 3 whether or not a word included in the correction result and its pronunciation set are registered, and selects a word and pronunciation set that are not registered. To do. If there is no corresponding pronunciation as a result of referring to the pronunciation dictionary database, the language model update unit 12 determines the word in the correction result so that the word is not used for n-gram estimated later. Replace with a symbol representing an unknown word.

(Step S53: n-gram calculation process)
The language model update unit 12 estimates n-gram from the words included in the correction result. For example, when trigram is considered as n-gram, the estimation formula is the following formula (17).

  Here, C (u, v) is the frequency of the word pair (u, v) in the correction result, and C (u, v, w) is the frequency of the word triplet (u, v, w). is there. P (w | u, v) is a trigram, and is a conditional probability that the word w occurs following the word duplication (u, v). The language model update unit 12 sequentially shifts the word triplet from the beginning of the correction result one word at a time, and performs the above calculation from all the word triplets of the correction result.

In the following, the language model updating unit 12, as shown in the following expression (18), currently stored in the language model storage unit 4, the linear interpolation and trigram referenced by the speech recognition unit 2, the trigram obtained in the above Connect by.

Here, P ^new (w | u, v) is an updated trigram, and P ^old (w | u, v) is a trigram of the language model currently stored in the language model storage unit 4. Further, ν is a weight for linear interpolation and is determined in advance.

(Step S54 : model update process)
The language model update unit 12 adds the combination of the word and pronunciation selected in step S52 to the pronunciation dictionary stored in the pronunciation dictionary storage unit 3. Furthermore, the language model updating unit 12, the trigram obtained in step S 53, replacing the language model currently stored in the language model storage unit 4.
Note that the update of the error correction model in the error correction model update unit 10 and the update of the language model in the language model update unit 12 operate independently of each other, so that both can perform model update at any utterance end timing. .

[5. effect]
According to the present embodiment, since the error correction model can be estimated with a small amount of calculation, the error correction model learning device 1 can sequentially update the error correction model used for speech recognition with low delay. By performing speech recognition using the error correction model that is sequentially updated, the speech recognition unit 2 can reduce errors in speech recognition by reflecting real-time characteristics as compared with the conventional case.

[6. Others]
Note that the error correction model learning device 1 described above has a computer system therein. The process of operation of the error correction model learning device 1 is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

1 error correction model learning device 2 speech recognition unit 3 pronunciation dictionary storage unit 4 language model storage unit 5 acoustic model storage unit 6 error correction model storage unit 7 speech recognition result correction unit 8 morpheme analysis unit 9 morpheme analysis dictionary database storage unit 10 error Modified model update unit 11 Storage unit 12 Language model update unit 13 Storage unit 14 Pronunciation dictionary database storage unit

Claims (5)

A voice recognition result correction unit for correcting the voice recognition result according to the input instruction;
Learning a word error tendency from a difference between a linguistic feature included in the speech recognition result and a linguistic feature included in the correction result by the speech recognition result correcting unit, and correct the word error in the speech recognition an error correction model update unit error correction model for correcting, updating according to a tendency of error in the words learned,
Equipped with a,
The linguistic characteristic is a frequency of a continuous word string or a part of speech string of continuous words;
The error correction model is a calculation formula for correcting a score of speech recognition using a feature function based on the linguistic feature and a feature weight of the feature function,
The error correction model update unit updates the feature weight of the error correction model according to a tendency of the learned error of the word;
An error correction model learning device characterized by the above. The error correction model update unit uses the frequency of co-occurrence of words or word parts of speech included in the speech recognition result and the frequency of co-occurrence of words or word parts of speech included in the correction result. Learn trends,
The error correction model learning device according to claim 1.   The speech recognition result correcting unit corrects the speech recognition result obtained sequentially for each block,
  The error correction model update unit sequentially corrects a word error based on a difference between a linguistic feature included in the speech recognition result in the block and a linguistic feature included in the correction result in the block. The tendency is learned, the feature weight in the block is calculated according to the learned error tendency of the word, and the feature weight in the calculated block and the feature weight calculated in the block before the block are calculated. Based on weighted addition, a process of updating the feature weight of the error correction model is performed.
  The error correction model learning apparatus according to claim 1, wherein the error correction model learning apparatus is a learning apparatus. A speech recognition unit for recognizing input speech and correcting and outputting an error in selection of a speech recognition result obtained from the input speech using the error correction model updated by the error correction model update unit; Prepare
The error correction model learning device according to any one of claims 1 to 3, wherein Computer
Speech recognition result correcting means for correcting the speech recognition result in accordance with the input instruction;
Learning the error tendency of a word from the difference between the linguistic feature included in the speech recognition result and the linguistic feature included in the correction result by the speech recognition result correcting means, and correct the word error in the speech recognition an error correction model updating means for error correction model for correcting, updating according to a tendency of error in the words learned,
Equipped with,
The linguistic characteristic is a frequency of a continuous word string or a part of speech string of continuous words;
The error correction model is a calculation formula for correcting a score of speech recognition using a feature function based on the linguistic feature and a feature weight of the feature function,
The error correction model update means updates the feature weight of the error correction model according to the tendency of the learned error of the word,
A program for functioning as an error correction model learning device.

Images