JP5749187B2 - Parameter estimation device, parameter estimation method, speech recognition device, speech recognition method and program - Google Patents

Description

  The present invention relates to a parameter estimation device and parameter estimation method for estimating a parameter for adjusting a probabilistic finite state model including parameters used in speech recognition, and a speech recognition device for obtaining a speech recognition result for speech data using the estimated parameter. And a speech recognition method and program.

In the following description, the symbols “^”, “ ⁻ ”, “ ^˜ ”, etc. used in the text should be written directly above the previous character. Immediately after. In the formula, these symbols are written in their original positions. Also, in the drawings used for the following description, the same reference numerals are given to components having the same functions and steps for performing the same processing, and redundant description is omitted.

  A speech recognizer is generally represented by a probabilistic finite state model. In the finite state model, a state variable having a discrete value transitions to a corresponding state by receiving an input, that is, a part of speech (hereinafter referred to as “frame”). The probabilistic finite state model is an extension of the finite state model using the concept of probability, and indicates a state in which a transition to a state is only given probabilistically when a frame is received. In the speech recognition apparatus based on the probabilistic finite state model, after receiving all the frame sequences, the state transition with the highest probability is estimated, and a word string corresponding to the state transition is output.

  In a speech recognition device based on WFST (Weighted Finite State Transducer), which has been widely used in recent years as a speech recognition device using a probabilistic finite state model, for example, a finite state expressing a probabilistic transition of a word Model, WFST that defines probabilistic conversion from a word to its pronunciation (phoneme sequence), WFST that defines conversion from phoneme to context-dependent phoneme considering the articulation of previous and next phonemes, depending on each context-dependent phoneme A final probabilistic finite state model is created by synthesizing each module necessary for speech recognition, such as a previously created hidden Markov model (hereinafter also referred to as “HMM”), using a WFST synthesis operation. Get. Generally, the parameters of each module are previously defined manually or learned from learning data.

  However, in the speech recognition apparatus obtained in this way, even if each module is optimal in each learning, the synthesized stochastic finite state model is not always optimal. For example, when considering the overall optimality of a kind of learning criterion called “discriminative learning criterion” for reducing the predicted speech recognition error rate, the optimal solution is not always found in learning in the form of decomposition into modules. It is known that there is no (see Non-Patent Document 1).

FIG. 1 shows a functional block diagram of a speech recognition apparatus 91 based on a probabilistic finite state model. In a speech recognition device based on a probabilistic finite state model, the speech recognition device is expressed by a transition cost function π (X ⁻ , s, s ′, t, t ′) and an output symbol o (s, s ′). Wherein X ^- is an input series, a feature vector x is extracted from the audio data input ^- are those arranged in time _{^{series, X - = {x - 1}} , x - 2, ..., x - T }. Here, T represents the size (total number of frames) of input audio data. Note that the input sequence X ⁻ is generally calculated in advance by the feature vector sequence calculation unit 7 as an input to the speech recognition device 91. The transition cost function [pi, when certain conditions s, input sequence X ^- 'if it is entered until (where, 1 ≦ t <t' from the time t time t ≦ T), the transition to the next state s' The difficulty of the event is defined, and the larger the value of the transition cost function π, the more difficult the transition from the state s to the state s ′. The output symbol o (s, s ′) indicates a word obtained as a result of speech recognition when the state transitions from s to state s ′. As the output symbol, for example, a word is conceivable. In addition, there is a state transition that does not generate a result of speech recognition. In this case, for convenience, it is considered that an epsilon symbol ε is output.

This transition cost function π and state sequence s ⁻ : = {s ₁ , s ₂ ,..., S _M }, segment time t ⁻ : = {t ₁ , t ₂ ,..., T _M } (where t ₁ = 1, The following sequence cost function Π is defined using t _M = T).

This series cost function Π is, (hereinafter also referred to as "state transition s → s'") state transition between from time t _m of time t _{m + 1} from the state s _m to the state s _{m + 1} is the transition cost function π that occurs, m = 1 to m = M−1 and represents the cost of the state sequence s ⁻ when the entire input sequence X ⁻ is input and the segment time t ⁻ is determined. Is. In the following, the outputs of the transition cost function π and the sequence cost function Π are also simply referred to as costs.

The shortest path search unit 911 of the speech recognition apparatus 91 outputs the optimum state series s ^˜ when the input series X ⁻ is input as a solution of the following shortest path problem using this series cost function Π.

Word string serving as the final recognition result, the optimal state sequence _{^{s ~ = {s ~ 1,}} s ~ 2, ...} output symbol o ^{(s ~} corresponding to each state transition _{^{s ~ m → s ~ m +}} 1 in _m , s ^˜ _{m + 1} ) can be represented by the enumeration except for ε. Output symbol extracting section 8 extracts the estimated word string from the optimal state sequence s ^~.

  In a speech recognition apparatus based on an HMM and a language model (N-gram language model / network grammar) that has been frequently used in the past, each transition cost function π is expressed in the following form.

However, ω (s, s ') is the state transition s → s' is a weight parameter for the input sequence X ^- is a state transition costs are designed independently. ω (s, s ′) may be given manually, or in speech recognition based on WFST, the cost of each state transition obtained by combining a plurality of WFSTs in advance may be used. I [s, s ′] is an index of the output distribution associated with the state transition s → s ′, and is obtained by an HMM learning device or the like. The output distribution P (x ⁻ _τ | I [s, s ′]) is modeled using, for example, a mixed Gaussian distribution for each I [s, s ′]. The state transition costs ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) related to these transition cost functions π are stored in the finite state model storage unit 912, and the shortest path search unit 911 described above. Used by.

However, in the definition of the transition cost function π as shown in Equation (3), the relationship between the state transition and the input sequence has been expressed only through the output distribution obtained by learning the hidden Markov model. That is, the conventional speech recognition apparatus 91 input sequence X ^- wherein included in the vector x ^- a part related to _{^{τ P (x - τ | I}} [s, s']) and a portion other than omega (s , S ′) are individually learned, and the relationship between all possible state transitions and input frames has not been explicitly defined. In Patent Document 1, this transition cost function π is converted to a feature vector φ ⁻ (X ⁻ , t, t ′ ₎ including an adjustment parameter vector α ⁻ _{h (s, s ′)} and an input sequence X ⁻ for each state transition s → s ′. ) To be expanded as follows to realize more advanced expressions and their simultaneous optimization.

However, “ ^T ” represents transposition. FIG. 2 shows a functional block diagram of the speech recognition apparatus 92 expressed by Expression (4). h (s, s ′) represents the state transition s → s ′. In the case of this expression, if h (s, s ′) is appropriately designed, a different adjustment parameter vector α ⁻ _{h (s, s ′)} is introduced for all state transitions s → s ′, and the adjustment parameter vector _{^{α - h (s, s '}} ) and the feature vector ^{φ - (X -, t,} t') inner product of is reflected in the transition cost function [pi. Patent Literature 1 exemplifies the following form as the feature vector φ ⁻ (X ⁻ , t, t ′).

By using this expression and appropriately learning the adjustment parameter vector α ^- _{h (s, s ′)} , it is possible to introduce a cost based on learning that takes the whole into consideration in addition to the conventional speech recognition apparatus. Here, in Patent Document 1, the Perceptron method is used to determine the adjustment parameter vector α ⁻ _{h (s, s ′)} .

The shortest path search unit 921 receives the input sequence X ⁻ and stores the adjustment parameter vector α ⁻ _{h (s, s ′)} stored in the finite state model adjustment parameter storage unit 923 and the finite state model storage unit 912. State transition costs ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) are taken out, and the optimum state sequence s is obtained based on the equations (1), (2), and (4). ^Are obtained and output to the output symbol extraction unit 8.

JP, 2011-164336, A

Jen-Tzung Chien, Chuang-Hua Chueh, "Joint acoustic and language modeling for speech recognition", Speech Communication, March 2010, Volume 52, Issue 3, p.223-235

In general, since it is difficult to completely reduce speech recognition errors, the accuracy of a speech recognition device is measured by an index such as a word error rate. However, the Perceptron method of Patent Document 1 and maximum mutual information (hereinafter “MMI”) are used. A conventional method such as a method (referred to as reference 1) is a method in which all word strings other than the correct word string are regarded as incorrect answers and learning is performed.
(Reference 1) S. Kapadia, V. Valtchev, SJ Young, "MMI training for continuous phoneme recognition on the TIMIT database", Proc. ICASSP, 1993, Vol. 2, pp. 491-494

  Empirically, it is said that it is better to use fine errors when measuring the error scale. In the method in which learning is performed by regarding all word strings other than the correct word string as incorrect answers, the accuracy of parameter estimation is not sufficient because the unit of error is large.

  The present invention provides a parameter estimation technique that enables more robust parameter estimation by obtaining an error in a smaller unit based on a fine-grained error criterion and finely considering the “degree of incorrect answer” among incorrect answers. The purpose is to do.

  In order to solve the above-described problem, according to the first aspect of the present invention, the parameter estimation device generates a probability finite state model including parameters used in speech recognition, learning data, and a correct speech recognition result of the learning data. A recording unit that stores correct state transition sequences that are corresponding state transition sequences and adjustment parameters that are parameters for adjusting the stochastic finite state model, and speech recognition for learning data using the stochastic finite state model And a recognition unit that generates a recognition state transition sequence that is a sequence of state transitions corresponding to the speech recognition result obtained as a result of performing and a detailed calculation that calculates an error measure based on the difference between the correct state transition sequence and the recognized state transition sequence. A granularity error scale calculation unit and a parameter estimation unit that corrects the adjustment parameter according to the error scale are included.

  In order to solve the above problems, according to the second aspect of the present invention, the parameter estimation method performs speech recognition on learning data using a stochastic finite state model including parameters used in speech recognition. A recognition step for generating a recognition state transition sequence corresponding to a speech recognition result obtained as a result, a correct state transition sequence corresponding to a correct speech recognition result of learning data, and a recognition state transition A fine-grained error measure calculating step for calculating an error measure based on a difference from the series, and a parameter estimating step for correcting an adjustment parameter that is a parameter for adjusting the probability finite state model according to the error measure.

  According to the present invention, there is an effect of improving the accuracy of parameter estimation.

The functional block diagram of the conventional speech recognition apparatus. The functional block diagram of the speech recognition apparatus of patent document 1. FIG. The functional block diagram of the parameter estimation apparatus which concerns on 1st embodiment. The figure which shows the processing flow of the parameter estimation apparatus which concerns on 1st embodiment. 5A is a correct lattice, and FIG. 5B is a recognition lattice. The figure for demonstrating the processing flow for producing _| generating array c ( _tau) . The figure for _{demonstrating} arrangement _| sequence _cτ . The functional block diagram of a parameter estimation part. The figure which shows the processing flow of a parameter estimation part. The figure which shows the simulation result of speech recognition.

  First, the points of the present invention will be described.

[Points of Invention]
For a learning method with higher accuracy than the Perceptron method used in Patent Document 1, the present invention introduces a fine-grained error criterion. The fine-grained error criterion has so far learned the hidden Markov model (logP (x ⁻ _τ | I [s, s ′]) in Equations (3) and (4)) and the state transition cost ω (s, s ′) Have been used (see references 2-4).
(Reference 2) D. Povey, PC Woodland, "Minimum phone error and I-smoothing for improved discriminative training", Proc. ICASSP, 2002, Vol. 1, pp. 105-108
(Reference 3) D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, K. Visweswariah, "Boosted MMI for model and feature-space discriminative training", 2008, Proc. ICASSP, pp. 4057-4060
(Reference 4) E. McDermott, S. Watanabe, A. Nakamura, "Discriminative training based on an integrated view of MPE and MMI in margin and error space", Proc. ICASSP, 2010, pp. 4894-4897

  Minimum phone error (hereinafter also referred to as “MPE”) method (refer to Reference 2), boosted MMI (hereinafter also referred to as “bMMI”) method (refer to Reference 3) and differenced MMI (hereinafter referred to as “dMMI”) (Referred to as reference 4) is a technique that enables more robust parameter estimation by carefully considering the “degree of incorrect answer” among incorrect answers. Empirically, it is said that it is better to use fine errors when measuring the error scale. From such a viewpoint, learning that minimizes phoneme errors is often performed during learning.

  (1) In order to further improve the accuracy of the speech recognition apparatus that synthesizes the separately learned modules, it is essential to learn the adjustment parameters after synthesis, as in Patent Document 1. (2) In addition, the existing technology considers whether or not the recognition result completely matches the given correct answer as a learning criterion. However, in actual speech recognition, the word error rate of speech recognition can be reduced. is important. (3) Furthermore, in related techniques in the hidden Markov model learning technique (see References 3 and 4), phoneme errors have been used as a fine-grained error measure, but the finest error of a speech recognition device based on a finite state model is It is a state transition error, and it seems important to use a finer error scale.

  Therefore, in the present invention, adjustment parameters are learned and estimated after synthesis, and at that time, fine-grained error criteria such as the MPE method are used to learn parameter vectors defined for each state transition of the speech recognition apparatus. Is used. For example, parameter learning is performed using a fine-grained error criterion based on how many times the state transition is mistaken.

  Hereinafter, embodiments of the present invention will be described. It should be noted that processing performed in units of elements of vectors and matrices is applied to all elements of the vectors and matrices unless otherwise specified.

<First embodiment>
FIG. 3 is a functional block diagram of the parameter estimation apparatus 100 according to the first embodiment, and FIG. 4 shows the processing flow. The parameter estimation device 100 includes a finite state model storage unit 101, a learning data storage unit 103, a recognition unit 105, a fine-grained error scale calculation unit 107, a correct path storage unit 109, a parameter estimation unit 111, and a finite state model adjustment parameter storage unit 113. including.

The parameter estimation apparatus 100 optimizes the adjustment parameter vector α ⁻ _{h (s, s ′)} using the learning input sequences X ^{− (1)} , X ^{− (2)} ,..., X− ^(N). . Various methods are conceivable as a method for realizing the present invention. In this embodiment, a method using a lattice based data structure and a learning criterion applying bMMI or dMMI will be described. The functional block diagram of the speech recognition device 12 that performs speech recognition using the adjustment parameter vector α ⁻ _{h (s, s ′)} learned by the parameter estimation device 100 has the same configuration as that of FIG. However, the method of generating the adjustment parameter vector α ⁻ _{h (s, s ′)} stored in the finite state model adjustment parameter storage unit 123 is different from that of Patent Document 1.

The finite state model storage unit 101 stores a stochastic finite state model including a state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) used in speech recognition. Prior to the estimation of the adjustment parameter vector α ⁻ _{h (s, s ′} ), the learning is finished for the state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]). Think about the situation. For example, the finite state model construction unit 65 takes out the HMM and language model obtained by the learning method of the conventional speech recognition apparatus from the HMM storage unit 61 and the language model storage unit 63, and the state transition cost ω (s, s ′). And logP (x ⁻ _τ | I [s, s ′]) are calculated and stored in the finite state model storage unit 101.

The learning data storage unit 103 stores learning data input sequences X- ⁽¹⁾ , X- ⁽²⁾ ,..., X- ^(N) .

The correct path storage unit 109, a learning data input sequence ^{X - (1), X -} (2), ..., X - respectively the correct speech recognition result of the ^(N) corresponding state transition sequence (hereinafter "correct S ^ ⁽¹⁾ , s ^ ⁽²⁾ , ..., s ^ ^(N) and segment times t ^ ⁽¹⁾ , t ^ ⁽²⁾ , ..., t ^ ^(N) are stored. Has been. The correct state transition sequence and the segment time can be easily obtained using an existing speech recognition apparatus by manually giving a word string that is a correct speech recognition result to the input sequence.

<Recognition unit 105>
The recognition unit 105 extracts the state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) of the stochastic finite state model from the finite state model storage unit 101, and uses these values. , X- ^(N) , which is the learning data, is a sequence of state transitions corresponding to a speech recognition result obtained as a result of performing speech recognition on the input sequence X- ⁽¹⁾ , X- ⁽²⁾ ,. (Referred to as “recognition state transition series”) (s 1) and output to the fine-grained error scale calculation unit 107 and the parameter estimation unit 111.

For one input sequence X- ⁽ⁿ⁾ (where n = 1, 2,..., N), all possible state transition sequences may be generated as recognition state transition sequences, A sequence cost function Π may be obtained based on 1) and (3), and a state transition sequence corresponding to the top R items having a smaller sequence cost function Π may be generated as a recognized state transition sequence. However, R is an integer of 1 or more. Therefore, the recognized state transition sequence includes only a state transition sequence that is easy to be mistaken, and may not include a correct state transition sequence.

In this embodiment, in order to perform learning that minimizes a state transition error, as preparation, a lattice corresponding to a correct state transition sequence and segment information, and input sequences X ^{− (1)} and X ^{− ( 2)} ,..., X- ^(N) are subjected to speech recognition processing, and a lattice recording a recognition state transition sequence obtained as a recognition result is prepared. Note that the lattice is a graph representing a connection between states (hereinafter also referred to as “arc”).

The lattice corresponding to the correct state transition sequence is called a correct lattice and is represented by L ^ ⁽ⁿ⁾ . The lattice corresponding to the recognition state transition sequence is called a recognition lattice and is represented by L ⁽ⁿ⁾ . FIG. 5A shows an example of a correct answer lattice, and FIG. 5B shows an example of a recognition lattice. The correct lattice can be uniquely generated from s ⁽ⁿ⁾ and t ⁽ⁿ⁾ . In the case of FIG. 5A, s ⁽ⁿ⁾ = {1, 13, 9, 14, 15, 1}, t ^{( n)} can be considered to be generated by enumerating them from {1, 3, 13, 17, 20, 25}. However, the correct lattice L ^ ⁽ⁿ⁾ is calculated in advance using a correct word string and a speech recognition device.

  In the present embodiment, the recognition unit 105 generates and outputs this recognition lattice as a recognition state transition sequence.

Each lattice can be mathematically understood as a set of arc sequences, and this idea will be used in the following description. In this view, the lattice variable L can be considered as a set (set) of all possible cases that enumerates a series of arcs that must pass from the beginning to the end. For example, the correct answer lattice in FIG. 5A is a set of one arc sequence {a ^ ₁ , a ^ ₂ , a ^ ₃ , a ^ ₄ , a ^ ₅ }, and the recognition lattice in FIG. 5B is the six arc sequences e shown below. it can be regarded as a set of _{1 ~e 6.}
e ₁ = {a ₁ , a ₂ , a ₃ , a ₄ , a ₅ }
e ₂ = {a ₁ , a ₇ , a ₁₀ , a ₁₁ , a ₅ }
e ₃ = {a ₁ , a ₇ , a ₁₀ , a ₁₂ , a ₁₃ }
e ₄ = {a ₆ , a ₉ , a ₃ , a ₄ , a ₅ }
e ₅ = {a ₆ , a ₈ , a ₁₀ , a ₁₁ , a ₅ }
e ₆ = {a ₆ , a ₈ , a ₁₀ , a ₁₂ , a ₁₃ }.
In the case of FIG. 5B, the number of arcs included in each arc series is the same, but a different number of arcs may be included for each arc series. Each arc a _i is associated with pre-transition state, post-transition state, pre-transition time and post-transition time information, and s (a _i ), s ′ (a _i ), t (a _i ) and t ′ (a _i ). For example, arc a ₂ in FIG. 5B is associated with s (a ₂ ) = 13, s ′ (a ₂ ) = 9, t (a ₂ ) = 3, and t ′ (a ₂ ) = 13. Further, the lattice may be given information such as an output symbol o (s, s ′) corresponding to the recognition result and a cost calculated at the time of recognition. Further, the state transition from the state s to the state s ′ is represented as (s (a _i ), s ′ (a _i )).

[Principle of parameter estimation]
Before describing the processing of the parameter estimation unit 111 and the fine grain error scale calculation unit 107, the principle of parameter estimation will be described. The target of this embodiment is the adjustment parameter vector α ⁻ _{h (s, s ′)} in Expression (4), and this set of adjustment parameter vectors α ⁻ _{h (s, s ′)} is represented by A: = {α ⁻ _I | ∀i}. Where, i is the index obtained by h (s, s'), α - the d-th dimension of the _i expressed as alpha _{i, d.} If all s and s ′ are designed so that h (s, s ′) takes different natural numbers, all state transitions can be handled. Further, by changing the design of h (s, s ′), the memory usage and the calculation amount may be saved.

(1) MMI method Before introducing the fine-grained error scale, consider applying the MMI method (see Reference 1), which is an existing method, to learning the set A. Considering that an equation similar to the MMI method proposed for acoustic model learning is performed on the state series, a method for obtaining the set A as a solution of the following optimization problem is derived.

Equation (6) shows that the probability that the state transition s ⁻ occurs according to a given segment time t ⁻ is proportional to exp (−Π | (s ^ ⁽ⁿ⁾ , t ^ ⁽ⁿ⁾ , X ^{− (n)} )). assuming the input sequence X ^- can be said to be learning that maximizes the posterior probability of the correct state transition probability after having observed the. By executing this optimization, the set A is adjusted so that correct answers are more likely to occur than incorrect answers. This objective function treats all state transition patterns other than correct answers equally, and A measure of how far the transition pattern is from the correct answer is not considered.

Sum symbol sigma _s-in the _{denominator, t-(where} subscript s-, t-each ^s -, t ^- represents a) is for all possible state transitions and all possible segments Time Although it is a sum, it is generally said that a large amount of calculation is required to obtain this sum. Therefore, for example, the above objective function is approximated as follows using a recognition lattice.

Here summation Σ _{a-∈L (n) (where} the subscripts a-∈L (n) is, a ^- represents a ∈L ⁽ⁿ⁾⁾ are all arcs may take in recognition lattice ^{L (n)} is the sum of the sequence (or sequences cost function smaller upper the R arc series of [pi), the sum sigma _{j (However,} a _j is the arc line a ^- shows the j-th arc) is the arc line a ^- a contained _Is the sum of arc a _j . Similarly, the MMI molecule is expressed by a correct lattice.

Here, the sum Σ _j (where a ^ _j represents the j-th arc of the arc sequence of the correct lattice) is the sum of the arcs a ^ _j included in the arc sequence of the correct lattice. In general, an accurate lattice corresponding to the correct state transition ＾ can be used, so the numerator term is not an approximation.

As shown in FIG. 5, each arc a _j (where j = 1, 2,..., J) has a pre-transition state s (a _j ), a post-transition state s ′ (a _j ), and a pre-transition time t ( a _j ) and the time t ′ (a _j ) after the transition are recorded, and it is designed so that the denominator can be efficiently approximated by following the arc transition. Such a recognition lattice can be obtained using a conventional speech recognition apparatus. Since the objective function is continuous and the derivative of the objective function is also continuous, optimization can be performed using the steepest gradient method. The parameter estimation unit 111 can reduce the amount of calculation and optimize the adjustment parameter vector at high speed by approximating the objective function using the recognition lattice.

(2) bMMI method In order to introduce the fine-grained error scale, the bMMI method (see Reference 3) is introduced. In the bMMI method, instead of simply maximizing the posterior probability of the correct sequence, the probability distribution is corrected so that a sequence with a large error measure is likely to appear, and then the posterior probability of the correct sequence is attempted to be maximized. By this modification, the parameters are adjusted so that a series with a large error measure is less likely to appear.

Specifically, the following modified objective function is used, which considers that the larger the error measure E (s ⁻ , t ⁻ ; s ^ ⁽ⁿ⁾ , t ^ ⁽ⁿ⁾ ) is, the smaller the sequence cost function is.

However, σ is an adjustable parameter and is generally adjusted using a tuning data set. By using this objective function, it is known that a parameter with a large error scale E (s ⁻ , t ⁻ ; s ⁽ⁿ⁾ , t ⁽ⁿ⁾ ) is less likely to cause an error (reference document 3). reference).

When the state transition error is used as the error measure E, the number of occurrences of the state transition error can be obtained as the state transition error measure E ¹ for each arc of each recognition lattice even after approximation by the recognition lattice.

  In addition, bMMI molecules can be expressed in a lattice as with MMI molecules.

  The fine granularity error measure is a measure necessary for expressing how far the entire output symbol sequence is from the correct answer more finely than in the prior art. Specifically, use the edit distance between the actual output symbol and the correct output symbol, or use the edit distance between the actual phoneme sequence and the correct phoneme sequence after decomposing the output symbol into phonemes. Can be. In any case, it is a scale indicating how much the actual motion pattern differs from the motion pattern when the speech recognition operation is performed as correct. It has been empirically revealed that it is effective to use an error scale with as fine a granularity as possible, such as a phoneme error rather than an output symbol error (see References 2 to 4). In the present embodiment, as an example, in the state transition which is the finest granularity operation in speech recognition based on the finite state transition model, how different the correct state transition pattern and the actual state transition pattern are are different. Consider expressing it by counting the number of times it was performed.

In this embodiment, since lattice representation is used, a state transition error scale (E ^to (a _j ) in the above equation) for each arc indicating how many state transition errors have occurred in each arc of the recognition lattice is required. The error measure E ^~ (a _j ) of the state transition for each arc is calculated using the array representation c ⁽ⁿ⁾ of the correct lattice. First, each element c ⁽ⁿ⁾ _{τ in} the array representation c ⁽ⁿ⁾ of the correct lattice represents a pair of a pre-transition state and a post-transition state at each time, and can be obtained by an algorithm as shown in FIG.

In this algorithm, a state transition (s () (s ()) that occurs when the frame (τ-th frame) is processed in the τ-element c ⁽ⁿ⁾ _τ of the array c ⁽ⁿ⁾ having the same number of elements as the number of frames of the input sequence. a ^ _j ), s' (a ^ _j )) is stored (s107-3). The processing of s107 is performed on all frames in which state transitions (s (a ^ _j ), s' (a ^ _j )) have occurred (s107-2, s107-4, s107-5). Further, the processing of s107-2 to s107-5 is performed for all state transitions (s (a ^ _j ), s' (a ^ _j )) (s107-1, s107-6, s107-7). This makes it possible to obtain a simple expression about the frame number t and the corresponding correct state transition sequence. FIG. 7 shows an array representation c ⁽ⁿ⁾ when the above processing is performed on the correct lattice in FIG. 5A.

Using this array representation, the error scale E ^~ (a _j ) of state transition for each arc can be expressed as follows.

However, δ (a, b) is a function called a Kronecker delta function, and takes 1 if a = b, and 0 otherwise.

This calculation formula calculates how many frames each arc a _j and the corresponding state transitions (s (a _j ), s ′ (a _j )) differ from the correct state transition obtained above. Specifically, regarding the start time (t (a _j )) to the end time (t ′ (a _j ) −1) of each arc, the state transitions (s (a _j ), s ′ (a _j ) represented by the arc )) And the frame representation c ^{(n) of} the correct state transition are calculated by the delta function and the sum. Therefore, it can be said that the number of times of performing different state transitions is counted, and the time ((number of times of performing different state transitions) × (time for one frame)) of performing different state transitions is calculated. It can be said that there is.

(3) dMMI Method The dMMI method (see Reference 4) attempts to maximize the following objective function in order to directly reduce the error measure.

However, the sum Σ _{s′−, t′−} in the denominator (where subscripts s′− and t′− represent s ′ ⁻ and t ′ ⁻ respectively) represents all possible state transitions and all This is the sum of the segment times that can be taken. The fractional part of Equation (12), for all possible state transitions and all possible segments time, s ^-, t ^- it occurs may probability P (s ^{-, t -)} of the represent. This objective function is the expected value for the probability P (s ⁻ , t ⁻ ) of the negative error measure E (s ⁻ , t ⁻ ; s ^ ⁽ⁿ⁾ , t ^ ⁽ⁿ⁾ ), which is maximized Doing this is equivalent to adjusting the parameters to directly reduce the error measure. According to Reference 4, it is known that the following form using the bMMI objective function can be used as an effective approximation of the objective function.

Here, σ ₁ and σ ₂ are adjustable parameters, and σ ₁ ≠ σ ₂ . In principle, the above optimization can be solved without direct approximation.

<Fine Grain Error Scale Calculation Unit 107>
The fine-grained error measure calculation unit 107 calculates an error measure based on the difference between the correct state transition sequence and the recognized state transition sequence (s3 in FIG. 4), and outputs the error measure to the parameter estimation unit 111. In the present embodiment, the number of times of different state transitions between the correct state transition sequence and the recognized state transition sequence is counted, and the number of times (or time corresponding to the number of times) is calculated as an error scale. For example, fine grained error measure calculator 107, the correct state transition sequence from correct path storage unit 109 ^{s ^ (n), s ^ (n)} corresponding to the segment time ^{t ^ (n) is} taken out, correct lattice L ^ ^(N) is generated. Also, the recognition lattice L ⁽ⁿ⁾ is received from the recognition unit 105. An array c ⁽ⁿ⁾ is generated by using the correct lattice L ^ ⁽ⁿ⁾ , an error measure E ¹ is calculated by Expression (11), and is output to the parameter estimation unit 111. The generated correct lattice is output to the feature vector generation unit 111c.

<Parameter estimation unit 111>
FIG. 8 shows a functional block diagram of the parameter estimation unit 111, and FIG. 9 shows a processing flow thereof.

The parameter estimation unit 111 corrects the adjustment parameter vector α ⁻ _{h (s, s ′)} according ^to the error measure E˜ (s5 in FIG. 4).

The parameter estimation unit 111 includes an adjustment parameter initialization unit 111a, a gradient vector initialization unit 111b, a feature vector generation unit 111c, an arc weight calculation unit 111d, a partial differential coefficient update unit 111e, an adjustment parameter update unit 111f, and a convergence determination unit 111g. Including.
(Adjustment parameter initialization unit 111a)
Adjustment parameter initialization unit 111a, a set of adjustment parameters vector A: = ^- | adjustment parameter vector of {α _i ∀i} α ^- each element alpha _{i, d} (adjustment parameter vector of _{i α} ^- _i d th element of ) Is initialized (s111-1), and the set A in which all the elements α _{i, d} of all the adjustment parameter vectors α ^- _i are initialized is stored in the finite state model adjustment parameter storage unit 113. In this embodiment, initialization is performed simply by substituting 0. In addition, initialization based on the equivalent transformation from Gaussian distribution, initialization based on the statistics of the data set, and the like can be considered.

(Gradient vector initialization unit 111b)
The gradient vector initialization unit 111b initializes the partial differential coefficient Δ _{i, d} corresponding to the d-th element α _{i, d} of the adjustment parameter vector α ^- _i (s111-2) and outputs it to the partial differential coefficient update unit 111e. To do. Here, in order to take the sum of the gradients calculated from each data, first, all elements of the gradient vector are initialized with zero.

(Feature Vector Generation Unit 111c)
The feature vector generation unit 111c receives the correct lattice L ^ ⁽ⁿ⁾ from the fine granularity error scale calculation unit 107. Furthermore, the feature vector generation unit 111c extracts the learning data X- ⁽ⁿ⁾ from the learning data storage unit 103, and calculates a feature vector corresponding to each arc of the correct lattice L ^ ⁽ⁿ⁾ and the recognition lattice L ⁽ⁿ⁾ ( s111-4), and outputs it to the partial differential coefficient updating unit 111e. φ ^ _{n, j, d} is _{the d-} th element of the feature vector corresponding to the jth arc of the nth correct lattice, and φ ⁻ _{n, j, d} is the jth arc of the nth recognition lattice. Is the d-th element of the feature vector corresponding to. The calculation differs depending on what kind of feature vector is used. For example, if the feature vector of Expression (5) is used, a vector φ ⁻ (X ^{− (n)} , The d-th element of t (a _j ), t ′ (a _j )) may be used.

(Arc weight calculator 111d)
The arc weight calculation unit 111 d extracts the state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) from the finite state model storage unit 101, and learns data from the learning data storage unit 103. X- ⁽ⁿ⁾ is extracted, the recognition lattice L ⁽ⁿ⁾ is received from the recognition unit 105, the adjustment parameter vector α ^- _i is extracted from the finite state model adjustment parameter storage unit 113, and the error measure E is output from the fine-grained error measure calculation unit 107. receiving ^~ the learning reference to calculate the corresponding arc weights gamma _{n, i, j} in accordance with (BMMI or dMMI or) (s111-5), and outputs the partial differential coefficient update unit 111e.

The bMMI arc weights γ _{n, i, j} are obtained using the Forward-Backward algorithm. First, the forward cost alpha _j of each arc a _j according to the following recursion formula.

Here, Pre (j) is an index set of arcs connected prior to the j-th arc a _j , that is, Pre (j) = {j ′ | s ′ (a _{j ′} ) = s (a _j )}, Which can be obtained based on the recognition lattice L ⁽ⁿ⁾ . K is a numerical value called a lattice smoothing coefficient, and is a coefficient that can be adjusted to improve the accuracy of a technique using approximation by lattice. The transition cost function π is expressed by using the state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]), learning data X− ⁽ⁿ⁾ , and the adjustment parameter vector α ⁻ _i. Calculate based on (4). Similarly, the backward cost β _{j is obtained} by the following recursive formula.

Here, Fol (j) is a set of arc indices connected after the j-th arc a _j , that is, Fol (j) = {j ′ | s (a _{j ′} ) = s ′ (a _j )}. Using α _j and β _j and the backward cost B = β ₁ at the first arc, γ _{n, j} is expressed as:

Arc weight gamma _{n, j} of dMMI _is, γ _{n, j} and gamma ^~ _n as a function of the sigma of _BMMI, when placed with _j (sigma), different σ ₂ ≠ σ ₁ with respect to twice the above calculation (formula By repeating (14) and (15)), the following is obtained.

(Partial differential coefficient update unit 111e)
Partial differential coefficient update unit 111e is correct from feature vector generating unit 111c lattice ^{L ^ (n),} feature vector phi _{^ n, j, d} and phi ^- receiving _{n, j,} and _d, recognition from the recognition unit 105 lattice ^{L ( n)} , the arc weight γ _{n, j} is received from the arc weight calculator 111d _, the gradient coefficient corresponding to the input n is added to all arcs, and the partial differential coefficient Δ _{i, d} is updated (s111-6). ) And output to the adjustment parameter update unit 111f.

  The above processing (s111-4 to s111-6) is performed on all the learning data (s111-3, s111-7, s111-8).

(Adjustment parameter update unit 111f)
The adjustment parameter update unit 111f receives the partial differential coefficient Δ _{i, d} and updates the adjustment parameter vector α _{i, d} using this Δ _{i, d} (s111-9), and the finite state model adjustment parameter storage unit 113 is stored. For example, when the steepest gradient method is used as a training method, updating is performed according to the following formula.

Here, η is a variable called a learning rate and needs to be set appropriately. This update rule differs depending on the optimization method used.

(Convergence determination unit 111g)
The convergence determination unit 111g extracts the adjustment parameter vector α _{i, d} from the finite state model adjustment parameter storage unit 113, performs the convergence determination (s111-10), and ends the learning program if it has converged. If not converged, a control signal is transmitted to each unit so as to repeat the processing of s111-2 to s111-9. Judgment methods include simply counting the number of loops, continuing as long as the speech recognition rate continues to improve using validation data, and when the objective function value is evaluated and the fluctuation becomes smaller than the threshold. There is a method to stop by.

The speech recognition apparatus 12 in FIG. 2 performs speech recognition using the adjustment parameter vector α _{i, d} generated in the parameter estimation apparatus 100 stored in the finite state model adjustment parameter storage unit 123, thereby improving its accuracy. Can be improved. The finite state model adjustment parameter storage unit 123 (see FIG. 2) stores the same information as the adjustment parameter vector α _{i, d} finally stored in the finite state model adjustment parameter storage unit 113 (see FIG. 3). Is stored.

<Experimental result>
In order to confirm the effectiveness of this embodiment, a large vocabulary speech recognition experiment was conducted. In this experiment, a method based on validation data was used in the convergence determination unit 111g. The feature vector series extraction unit 7 converts the speech signal into 12-dimensional Mel-frequency cepstral coefficients (MFCC) and logarithmic power, and then combines the time differential value and the time second-order differential value of these 13-dimensional variables. Thus, a device for converting into a 39-dimensional input vector was used. The data set was English lecture audio data. The learning data set used 101 hours of data from lecture audio data. The number of series included in the learning data set is 60392, and the number of words is 1,076,647 words. The data set for calculating the error rate was 7.8 hours of data from lecture audio data. The number of series included in the evaluation data set is 6989, and the number of words is 74823 words. The results are shown in FIG. “+” In the table indicates that an adjustment parameter is additionally introduced to the speech recognition apparatus 91 individually learned by the dMMI method. The highest score of the speech recognition device 12 that performs speech recognition using the adjustment parameter vector α ⁻ of this embodiment is 27.1%, compared to the highest score of the speech recognition device 91 (dMMI method, 28.2%) ( In the case of bMMI method, σ2.0), it was possible to achieve an accuracy improvement of 1% or more compared to the prior art. In addition, even when compared with the speech recognition device 92 of Patent Document 1, an accuracy improvement of 0.7% was confirmed. From the above results, it can be said that the parameter estimation apparatus of the present embodiment functions effectively.

<Effect>
With such a configuration, the accuracy of parameter estimation can be improved. In this embodiment, in the speech recognition apparatus constructed by such individual learning of each module, the constructed state transition cost ω (s, s ′) and logP (x ⁻ _τ | I [s, s ′]) By re-learning while taking the whole into consideration, a more accurate parameter is estimated. A similar attempt is made in Patent Document 1, and the speech recognition apparatus of the present embodiment is the same as that described in Patent Document 1, but the adjustment parameter acquisition method used therein is disclosed in Patent Document 1. Is different. The accuracy of speech recognition can be further improved by using the adjustment parameter estimated according to the present embodiment.

<Other variations>
In the present embodiment, the correct lattice L ^ ^{(n) is obtained} by the fine-grained error scale calculating unit 107. However, using the correct word string and the speech recognition apparatus, the correct lattice L ^ ⁽ⁿ⁾ is obtained in advance by the correct lattice generating unit (not shown). You may store in the memory | storage part which does not.

  In this embodiment, the case where one arc sequence exists for one correct lattice has been described. However, a configuration in which one or more arc sequences exist for one correct lattice may be used. In that case, the MMI molecule is represented by the following formula instead of the formula (8).

Here the sum Σ _{a~∈L ^ (n) (where} subscripts a~∈L ^ (n) represents the ^{a ~} ∈L ^{^ (n))} are included in the correct lattice ^{L ^ (n)} is the sum of the arc sequence, the sum sigma _{j (However,} a ^ _j represents the j-th ^arc-arc sequence a) is the sum of the arc a ^ _j contained in the arc sequence a ^~.

In the present embodiment, only the arc weight γ _{n, j} corresponding to the recognition lattice is calculated, but the arc weight γ ^ _{n, j} corresponding to the correct lattice may be calculated. In this case, the arc weight calculation unit 111d receives the correct answer lattice from the feature vector generation unit 111c, and uses the following formula instead of the formulas (14) to (16) to calculate the arc weight γ ^ _{n, j} of the bMMI. calculate.

As with this case error measure E ^~ this embodiment, it is necessary to calculate using the correct lattice that contains only one correct answer.

The arc weight γ _{n, j} of dMMI is obtained by the following equation instead of equation (17).

In this case, the partial differential coefficient updating unit 111e calculates the partial differential coefficient Δ _{i, d} using the following expression instead of the expression (18).

Note that the shape of the feature vectors φ ^ and φ ⁻ is not limited to the form expressed by the equation (5), as in the case of Patent Document 1.

  There are various types of methods for constructing a stochastic finite state model for speech recognition, but the present invention can be applied to a "speech recognition device based on a stochastic finite state model". It can also be applied to a stochastic finite state model. In other words, the present invention can be applied by transforming a speech recognition apparatus that has been realized by various methods into an abstract form of a speech recognition apparatus based on a probabilistic finite state model. Almost all conventional speech recognition devices can be abstracted in the form of speech recognition devices 91 and 92 of the prior art, and the present invention can be applied to increase the accuracy of speech recognition devices that can be abstracted in this shape. is there. Therefore, most of the speech recognition devices that are currently mainstream can be expanded by the present invention.

In the present embodiment, the convergence determination unit 111g performs convergence determination, and when the convergence is not achieved, the processing of s111-2 to s111-9 is repeated (see FIG. 9). At this time, the recognition processing (FIG. 4) is performed. It is good also as a structure which repeats the process after s1). In this case, the recognizing unit 105 extracts the updated adjustment parameter vector α ⁻ _{h (s, s ′)} from the finite state model adjustment parameter storage unit 113 (indicated by a broken line in FIG. 3), and based on Expression (4). A recognition lattice L ⁽ⁿ⁾ is generated. At this time, the adjustment parameter initialization process (s111-1 in FIG. 8) is omitted in the second and subsequent iterations.

  In the present embodiment, the number of state transition errors is used as an error measure for the sake of simplicity. However, other error measures may be used, and the present invention does not simply separate the correct answer from the incorrect answer, but determines the distance between them. It can be applied to all methods that are used in detail.

  The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
The parameter estimation device and the speech recognition device described above can be functioned by a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a process procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

DESCRIPTION OF SYMBOLS 12 Speech recognition apparatus 100 Parameter estimation apparatus 101 Finite state model storage part 103 Learning data storage part 105 Recognition part 107 Fine granularity error scale calculation part 107 Fine granularity error scale calculation part 109 Correct answer path storage part 111 Parameter estimation part 111a Adjustment parameter initialization Unit 111b gradient vector initialization unit 111c feature vector generation unit 111d calculation unit 111e partial differential coefficient update unit 111f adjustment parameter update unit 111g convergence determination unit 113 finite state model adjustment parameter storage unit

Claims (11)

A stochastic finite state model including parameters used in speech recognition, learning data, a correct state transition sequence that is a sequence of state transitions corresponding to a correct speech recognition result of the learning data, and for adjusting the stochastic finite state model A recording unit for storing adjustment parameters as parameters;
A recognition unit that generates a recognition state transition sequence that is a sequence of state transitions corresponding to a speech recognition result obtained as a result of performing speech recognition on the learning data using the stochastic finite state model;
Based on the difference between the correct state transition series and the recognized state transition series, a fine-grained error scale calculator that calculates an error scale in units of state transition errors ;
Including a parameter estimation unit that corrects the adjustment parameter according to the error scale,
Parameter estimation device. The parameter estimation device according to claim 1,
The fine-grained error scale calculation unit counts the number of times different state transitions are performed between the correct state transition sequence and the recognized state transition sequence, and the number of times or a time corresponding to the number of times is the error As a measure,
Parameter estimation device.   The parameter estimation device according to claim 2,
  The correct state transition sequence and the recognized state transition sequence are expressed by a lattice that represents a connection between the states, and each arc of the lattice is represented by a _j And arc a _j The state before transition, the state after transition, the time before transition, and the time after transition of s (a _j ), S '(a _j ), T (a _j ) And t '(a _j ) And arc a _j State transition corresponding to (s (a _j ), S '(a _j )), And each element of the lattice array expression corresponding to the correct state transition sequence is c ^(N) _τ And the delta function of Kronecker is δ and the arc a _j E ~ (a _j ), And the fine-grained error scale calculation unit

To calculate the error measure,
  Parameter estimation device. The parameter estimation device according to any one of claims 1 to 3 ,
The correct state transition series and the recognized state transition series are represented by a lattice that represents a connection between the states,
The recognition unit generates a recognition lattice indicating a recognition state transition sequence,
The fine-grained error scale calculation unit calculates an error scale based on a difference between the correct lattice corresponding to the correct state transition series and the recognition lattice,
The parameter estimation unit corrects the adjustment parameter according to the error scale using the correct lattice and the recognition lattice.
Parameter estimation device. Using the adjustment parameter estimated by the parameter estimation apparatus according to any one of claims 1 to 4, the speech recognition device for determining the speech recognition result for the speech data. A recognition step for generating a recognition state transition sequence that is a sequence of state transitions corresponding to a speech recognition result obtained as a result of performing speech recognition on learning data using a stochastic finite state model including parameters used in speech recognition; and ,
A fine-grained error scale calculation step for calculating an error scale in units of state transition errors based on a difference between a correct state transition series that is a state transition series corresponding to a correct speech recognition result of the learning data and the recognized state transition series When,
A parameter estimation step of correcting an adjustment parameter that is a parameter for adjusting the stochastic finite state model according to the error measure,
Parameter estimation method. The parameter estimation method according to claim 6 , wherein
In the fine-grained error scale calculation step, the number of times of performing different state transitions between the correct state transition sequence and the recognized state transition sequence is counted, and the number of times, or the time corresponding to the number of times is the error As a measure,
Parameter estimation method.   The parameter estimation method according to claim 7, comprising:
  The correct state transition sequence and the recognized state transition sequence are expressed by a lattice that represents a connection between the states, and each arc of the lattice is represented by a _j And arc a _j The state before transition, the state after transition, the time before transition, and the time after transition of s (a _j ), S '(a _j ), T (a _j ) And t '(a _j ) And arc a _j State transition corresponding to (s (a _j ), S '(a _j )), And each element of the lattice array expression corresponding to the correct state transition sequence is c ^(N) _τ And the delta function of Kronecker is δ and the arc a _j E ~ (a _j ), And the fine-grained error scale calculation step includes

To calculate the error measure,
  Parameter estimation method. The parameter estimation method according to any one of claims 6 to 8 ,
The correct state transition series and the recognized state transition series are represented by a lattice that represents a connection between the states,
In the recognition step, a recognition lattice indicating a recognition state transition sequence is generated,
In the fine-grain error scale calculation step, an error scale is calculated based on the difference between the correct lattice corresponding to the correct state transition sequence and the recognition lattice,
In the parameter estimation step, using the correct answer lattice and the recognition lattice, the adjustment parameter is corrected according to the error measure.
Parameter estimation method. Using the adjustment parameter estimated by the parameter estimating method according to any of claims 6 9, the speech recognition method for obtaining the speech recognition result for the speech data. Parameter estimation apparatus according to any one of claims 1 to 4, or a program for causing a computer to function as a speech recognition apparatus according to claim 5, wherein.

Images