JP4881357B2 - Acoustic model creation apparatus, speech recognition apparatus using the apparatus, these methods, these programs, and these recording media - Google Patents

Description

  The present invention relates to an acoustic model creation device that creates an acoustic model used at the time of speech recognition by a sequential adaptation method, a speech recognition device that performs speech recognition using an acoustic model created by the device, these methods, and these The present invention relates to a program and these recording media.

[voice recognition]
A functional configuration example of a conventional speech recognition apparatus is shown in FIG. 1, and a main flow of processing of the conventional speech recognition apparatus is shown in FIG. The speech recognition apparatus 2 mainly includes a feature extraction unit 4, a word string search unit 6, an acoustic model storage unit 8, and a language model storage unit 10.

  First, the acoustic model in the acoustic model storage unit 8 is read (step S2). In some cases, in addition to the acoustic model, a word model, a context-dependent phoneme model, and the like are read. Further, the language model storage unit 10 is read (step S4). The input recognition speech data is read into the speech recognition apparatus 2 (step S6), the recognition speech data is input to the feature extraction unit 4, and the recognition speech data is MFCC (Mel filter) for each frame (fixed time interval). It is converted into an acoustic feature quantity series (hereinafter referred to as “feature quantity series”) such as a bank cepstrum coefficient) vector (step S8). The converted feature quantity series is not shown in the figure, but is temporarily stored in the feature quantity storage unit. The stored feature quantity series is read out and input to the word string search unit 6.

  In the word string search unit 6, a score is calculated for the feature amount series of the recognition speech data using the acoustic model in the acoustic model storage unit 8, and the score for the language model or the like in the language model storage unit 10 is referred to this. A word string search is performed (step S10). In some cases, phoneme string search or isolated word search is performed. Finally, the recognition result is output as a word string (step S12). In some cases, only a phoneme string or an isolated word is output.

[Acoustic model creation]
Next, a method for creating an acoustic model will be described. The acoustic model is obtained by modeling the acoustic features of speech, and the speech data is converted into symbols such as phonemes and words by referring to the recognition speech data and the acoustic model. Therefore, the creation of the acoustic model greatly affects the performance of the speech recognition device. Usually, in the acoustic model for speech recognition, each phoneme is represented by a Left to right hidden Markov model (HMM), and the output probability distribution of the HMM state is represented by a mixed Gaussian distribution model (GMM). Therefore, what is actually stored in the storage unit as an acoustic model is that the HMM state transition probability a, the GMM mixture weight factor w, and the average vector parameter μ of the Gaussian distribution in the acoustic model in each symbol such as a phoneme. , And the covariance matrix parameter Σ of the Gaussian distribution in the acoustic model. These are referred to as acoustic model parameters, and the set is θ. That is, θ = {a, w, μ, Σ}. Accurately obtaining the value of the acoustic model parameter θ is an acoustic model creation process, and this process is called an acoustic model creation method.

  In recent years, an acoustic model is created by learning an acoustic model parameter θ from a large amount of speech data and information of a teacher label by a probabilistic statistical method. For normal learning data, teacher label information indicating which part is which phoneme is given. When the teacher label information is not given, the teacher label information is given by listening to the actual person and attaching the teacher label information or by using a voice recognition device. In the following description, it is assumed that the teacher label information is given to the learning voice data by the method described above.

  An example of the functional configuration of a conventional acoustic model creation device is shown in FIG. 3, and the main flow of processing of the conventional acoustic model creation device is shown in FIG. In FIG. 3 and FIG. 4, the provision of teacher label information is omitted.

  The acoustic model creation device 11 includes a feature extraction unit 4, a feature amount storage unit 5, and an acoustic model parameter learning unit 12. The learning voice data is read by the acoustic model creation device 11 (step S22). The read learning speech data is converted into a feature amount series by the feature extraction unit 4 (step S24). The converted feature quantity series is stored in the feature quantity storage unit 5. The stored feature quantity series is read out and input to the acoustic model parameter learning unit 12. If the teacher label does not exist (step S26), the teacher label information is given by the voice recognition device or manually (step S28).

  Next, learning of acoustic model parameters by the acoustic model parameter learning unit 12 will be described. From the data corresponding to each phoneme in the learning data obtained from the teacher label information, the acoustic model parameter θ (the HMM state transition probability a, the GMM mixture weight factor w, the GMM average vector parameter μ, and the covariance matrix parameter Σ ) Is called acoustic model parameter learning. There is a maximum likelihood learning method as a parameter learning method. Other acoustic model parameter learning includes Bayesian learning, discriminative learning, neural network, and the like.

  The acoustic model parameter learning unit 12 learns acoustic model parameters using the teacher label information corresponding to the voice data prepared in advance in the teacher label storage unit 14 (step S30). The acoustic model created by the acoustic model creation device 11 is output (step S32). If a teacher label is present in step S26, the process proceeds directly to step S30.

  Since the acoustic model parameters have millions of degrees of freedom, a large amount of learning speech data for several hundred hours is required to learn them. However, it is impossible to collect a large amount of speech data including all acoustic variation factors such as a speaker, noise, and speech style in advance and sufficiently learn millions of parameters. Therefore, adaptive learning is a very important technique as a method for estimating acoustic model parameters from a small amount of learning speech data.

[Adaptive learning based on conversion of acoustic model parameters]
Adaptive learning with respect to acoustic model parameters is a method of learning with a small amount of data using the initial model as a priori knowledge when the amount of learning data per parameter is small. The difference from the normal learning method is that an acoustic model is constructed using not only learning data but also an initial model. A learning method for constructing a new acoustic model from the initial model and learning data in this way is called adaptive learning.

In adaptive learning, attention is generally paid to the conversion between the initial acoustic model parameter θ ₀ and the newly created acoustic model parameter θ. For example, when a feature quantity sequence O = {o ₁ , o ₂ ,..., O _N | o _n ∈R ^D } expressed by N D-dimensional feature vectors is considered, estimation of the acoustic model parameter θ is considered. Instead, the conversion parameter estimation method considers the conversion parameter. That is, when the acoustic model parameter θ after adaptation is obtained as θ = f (θ ₀ , O) from the parameter θ _{0 of the} initial model and the feature amount series O, f (·) is obtained, and thereby the acoustic model parameter θ is newly obtained. It is a technique to obtain.

If f (•) is expressed parametrically (the relationship is expressed by a mathematical expression), adaptive learning initially estimates a conversion parameter W (which will be described in detail later) that is a parameter of f (•). It is obtained from the model parameter θ ₀ and the adaptation audio data O. This is called adaptive learning based on acoustic model parameter conversion.

[Linear regression method]
In adaptive learning, a method for estimating a linear regression matrix for an average vector parameter μ of a Gaussian distribution in an acoustic model is widely used (see Non-Patent Documents 1 and 2). FIG. 5 shows an example of the functional configuration of the acoustic model creation apparatus when a linear regression matrix is used, and FIG. 6 shows the main processing flow of the acoustic model creation apparatus in this case. The acoustic model creation device 21 using this method is composed of a feature extraction unit 4, a feature amount storage unit 5, and a parameter adaptation unit 22, which are a conversion parameter estimation unit 24, a conversion parameter storage unit. 26 and a model parameter conversion unit 28.

First, the initial acoustic model parameter θ ₀ is read into the initial acoustic model parameter storage unit 30 (step S40). Then, the adaptation audio data 20 is read (step S42), input to the feature extraction unit 4, and converted into the feature amount series O (step S44). The converted feature quantity series O is temporarily stored in the feature quantity storage unit 5. The stored feature quantity series O is input to the conversion parameter estimation unit 24. The processes of the conversion parameter estimation unit 24 and the model parameter conversion unit 28 will be described below.

An average vector parameter μ ₀ of a certain Gaussian distribution in the initial acoustic model parameter θ ₀ is linearly converted by the following equation (1).
μ = Aμ ₀ + ν (1)
Here, A is a D × D matrix, and is a matrix for rotating and expanding / contracting the average vector parameter μ ₀ . ν is a D-dimensional vector and represents a vector that translates the average vector parameter μ ₀ . At this time, the conversion parameter W = (ν, A). The transformation parameter W is efficiently obtained from the feature amount series O by repeated calculation using an expected value maximization (Expectation Maximization) algorithm (hereinafter referred to as EM algorithm) or a kind of MLLR (Maximum Likelihood Linear Regression) algorithm (step S46). ). The number of conversion parameters W to be estimated is D ² + D = D (D + 1). This is because the number of elements of the matrix A is D ² and the number of elements of the vector ν is D. Although the number of parameters is larger than the number of parameters D of the average vector, it is possible to reduce the number of parameters to be estimated by sharing the same transformation parameter with a plurality of Gaussian distributions. The estimated conversion parameter W is temporarily stored in the conversion parameter storage unit 26.

The stored conversion parameter W is input to the model parameter conversion unit 28. The model parameter conversion unit 28 obtains a new average vector parameter μ from the above equation (1) based on the obtained conversion parameter W and the initial average vector parameter μ ₀ (in the initial acoustic model parameter θ ₀ ) (step S1). S48). The average vector parameter μ is output as the acoustic model parameter μ (step S50).

[Sequential adaptation based on acoustic model parameter conversion]
Up to this point, adaptive learning for a group of feature amount sequences O = {o ₁ , o ₂ ,..., O _n ,..., O _N } (where N is the number of frames) has been considered. However, the acoustic characteristics of speech are greatly changed from time to time due to external factors such as noise and internal factors such as voicing. In order to follow such changes, it is effective to use sequential adaptive learning in which a sequential model is adapted to a large amount of speech data given in time series. At this time, let us consider adaptation in the case where a plurality of groups are given in a time series, instead of taking a feature quantity series as one group. In other words, the following equations (2) and (3) are considered.

However, t indicates the previous time, t + 1 indicates the current time, T in Equation (3) indicates the total number of _t , and θ _{t + 1} and θ _t are the current and previous acoustic model parameters. In this case, the acoustic model parameter theta _{t + 1} at a certain chunks t + 1 is determined from the coherent O _{t + 1} of the acoustic model parameters theta _t and the feature sequence obtained in the previous chunk t. That is, the acoustic model can be obtained from time to time by expressing it with the recurrence formula shown in the following formula (4). This is called a sequential adaptation method based on parameter conversion.
θ _{t + 1} = f (θ _t , O _{t + 1} ) (4)

FIG. 7 shows a procedure for converting acoustic model parameters when the sequential adaptation method is used. First, the acoustic model parameter θ ₁ is obtained by the model parameter conversion unit 28 using the feature amount series O ₁ and the initial acoustic model parameter θ ₀ . Next, the acoustic model parameter θ ₂ is obtained using the acoustic model parameter θ ₁ and the next feature amount series O ₂ . In this way, the current acoustic model parameter θ _{t + 1} is obtained using the previous acoustic model parameter θ _t and the current feature amount series O _{t + 1} .

[Linear regression method]
At this time, the application of the transformation parameter estimation method to the sequential adaptation is considered (see Non-Patent Document 2). The conversion parameter W is estimated from all the feature amount sequences earlier. However, in the sequential adaptation, W is estimated for each group (every t). If it is assumed that W _t = {ν _t , A _t }, the average parameter update formula (shown in the formula (4)) in the iterative adaptive method based on the parameter transformation is based on the formula (1) below. It can be expressed by a recurrence formula as shown in Formula (5).
μ _{t + 1} = A _{t + 1} μ _t + ν _{t + 1} (5)
As a result, sequential adaptation based on parameter conversion is realized. In the following description, At _{+ 1} is referred to as “a coefficient matrix when linearly expressing the average stochastic dynamics in the current acoustic model parameter”, and ν _{t + 1} is “average stochastic dynamics in the current acoustic model parameter. It is called "coefficient vector when linearly expressed".

The above-described successive adaptation method is obtained by the acoustic model parameters θ ₀ ,. . . , Θ _{t + 1} does not take into account how much estimation error is included. For this reason, when there is voice data that adversely affects learning, or when learning fails, the influence directly appears in the recognition performance, resulting in low robustness.

[Sequential adaptation method based on distribution transformation]
Next, the “sequential adaptation method based on distribution transformation” which is the basic concept of the present invention will be described. In this method, the estimation of the acoustic model parameter θ _t itself is not considered, but the distribution p (θ _t ) of the acoustic model parameter is considered (see Patent Document 1, Non-Patent Documents 3 and 4).

Thereby, the error by estimation can be considered from the variance of the distribution, for example. Further, consider a posterior probability distribution when a feature amount series O ^t = {O ₁ , O ₂ ,..., O _t } accumulated as an acoustic model parameter distribution is given. That is, instead of p (θ _t ), p (θ _t | O ^t ) is an estimation target. Here, O ^{t + 1} and O ^t indicate feature quantity sequences accumulated up to this time and the previous time.

Here, p (α | β) is a posterior probability (conditional probability) that is a probability that another event α occurs under the condition that a certain event β occurs. That is, p (θ _t | O ^t ) indicates a posterior probability that the acoustic model parameter is θ _t when the feature amount sequence O ^t is given. Thereby, since the information of the accumulated feature amount sequence O ^t can be added to the acoustic model parameter, robustness can be ensured. Accordingly, the recurrence formula p (θ _{t + 1} | O ^{t + 1} ) = F [p (θ _t | O ^t )] shown in the following formula (8) (8)
Is used to describe the recursion formula corresponding to the time evolution, that is, the change of the acoustic feature of the speech, so that the posterior probability distribution p of the acoustic model parameter, not the acoustic model parameter θ noted in the above formula (4), is used. Sequential adaptation based on (θ | O) can be realized. Here, F [•] is a functional having p (θ | O) as an argument. F [•] is expressed based on a partial feature quantity sequence of the feature quantity series O ^{t + 1} accumulated up to this time. In the following description, it is assumed that F [•] is expressed based on the feature amount series O ^{t + 1} accumulated up to this time. At this time, F [•] is expressed parametrically, and the conversion parameter W is appropriately estimated from, for example, the feature amount O _t, thereby realizing the sequential adaptation expressed by the equation (8). However, the estimation of the conversion parameter may use not only the feature quantity O _t but also a part of the feature quantity series O ₁ , O ₂ ,..., O _t , or the feature quantity series O ^t. .

Comparing the equation (4) and the equation (8), it can be understood that the equation (8) does not sequentially convert the parameters, but sequentially converts the posterior probability distribution. In addition, if the parameters of the posterior probability distribution p (θ ^_t │O _t) at time t and ω _t, the sequential update of p (θ ^_t │O _t) can be expressed in a sequential updating of the parameters ω _t. Accordingly, the adaptation can be realized sequentially by obtaining the posterior probability distribution parameter ω _t every moment. Therefore, in the successive adaptation method based on the distribution transformation, the posterior probability distribution parameter ω _t is updated instead of the posterior probability distribution p (θ _t | O ^t ).

  FIG. 8 shows an example of the functional configuration when the sequential adaptation is applied, FIG. 9 shows the order of sequentially adapting the parameter ω of the posterior probability distribution, and FIG. 10 shows the main processing flow. The acoustic model creation device 48 shown in FIG. 8 includes a feature extraction unit 4, a feature amount storage unit 5, and a model adaptation unit 50. The model adaptation unit 50 includes a sequential learning unit 52 and a posterior probability distribution storage unit 54. , The model update unit 56.

First, the parameter ω _t of the previous posterior probability distribution is read by the model adaptation unit 50 (step S60). Next, the audio data for adaptation is read (step S62), and the audio data for adaptation is input to the feature extraction unit 4 and converted into the feature amount series O _{t + 1} (step S64). The converted feature quantity sequence O _{t + 1} is temporarily stored in the feature quantity storage unit 5 and sequentially input to the learning unit 52.

In the sequential learning unit 52, the posterior probability distribution p (θ _t | O ^t ) of the acoustic model parameter obtained last time in consideration of the feature amount series accumulated up to the previous time, as in the equation (8), and the current time Based on the extracted feature quantity sequence O _{t + 1} , the posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) of the current acoustic model parameter adapted to the current feature quantity series is obtained (step S68). Hereinafter, a more specific method for obtaining the posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) by the sequential learning unit 52 will be described.

ここで式（９）の右辺にあるｐ(θ_ｔ＋１｜Ｏ^ｔ)はｐ(θ_ｔ｜Ｏ^ｔ)を用いると次のように表現される。
ｐ(θ_ｔ＋１｜Ｏ^ｔ)＝∫ｐ(θ_ｔ＋１｜θ_ｔ，Ｏ^ｔ)ｐ(θ_ｔ｜Ｏ^ｔ)ｄθ_ｔ （１０）
従って式（１０）を式（９）に代入することにより次式（１１）のような漸化式を導出することができる。

The function F [•] in the above equation (8) describing the time evolution from p (θ _t | O ^t ) to p (θ _{t + 1} | O ^{t + 1} ) can be given an arbitrary form. You can think of transformation. In this embodiment, as one specific function system, a recurrence formula derived theoretically without approximation from a probability product formula and Bayes' theorem is introduced. _First , p (θ _{t + 1} | O ^{t + 1} ) is expressed as follows from Bayes' theorem.

Here, p (θ _{t + 1} | O ^t ) on the right side of Expression (9) is expressed as follows using p (θ _t | O ^t ).
p (θ _{t + 1} | O ^t ) = ∫p (θ _{t + 1} | θ _t , O ^t ) p (θ _t | O ^t ) dθ _t (10)
Accordingly, a recurrence formula such as the following formula (11) can be derived by substituting formula (10) into formula (9).

式（１２）は、逐次的ではなく、与えられた適応用データＯ_１からｐ（θ_１│Ｏ_１
）を推定する通常の適応を示している。つまり、本発明は逐次適応のみならず通常の適応においてもその効果を与えることができる。 The right side of equation (11) includes a previous posterior probability distribution at (time ^{_{t) p (θ t │O t}} ), from p (θ ^_t │O _t) at the current (the next time t + 1) This is an equation for obtaining the posterior probability p (θ _{t + 1} | O ^{t + 1} ). Therefore, Expression (11) is called a recurrence expression of the posterior probability distribution of acoustic model parameters. By using this recurrence formula, the sequential learning unit 52 performs the sequential estimation of the posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) of the acoustic model parameter, which is accumulated up to the previous time and includes the information of the feature amount sequence O ^{t + 1.} Can be done. Further, the integral calculation of Expression (11) can be solved by numerical calculation such as Monte Carlo method. Here, time development is limited to the first step. That is, if t → 0 and t + 1 → 1, the following equation (12) is obtained.

Equation (12) is not sequential, but given adaptation data O ₁ to p (θ ₁ | O ₁
) Shows the normal adaptation to estimate. That is, the present invention can provide the effect not only in the sequential adaptation but also in the normal adaptation.

In order to realize the sequential adaptation according to the equation (11), the right side is the following four probability distributions p (O _{t + 1} | O ^t ), p (θ _t | O ^t ), p (O _{t + 1} | θ _{t + 1} , O ^t ), p (θ _{t + 1} | θ _t , O ^t ) need to be given a specific system. Here, p (O _{t + 1} | O ^t ) does not depend on the argument θ _{t + 1} of p (θ _{t + 1} | O ^{t + 1} ), which is the distribution to be obtained, and can be treated as a normalization constant. good. Consider the remaining three, p (θ _t | O ^t ), p (O _{t + 1} | θ _{t + 1} , O ^t ), and p (θ _{t + 1} | θ _t , O ^t ).

p (θ _t | O ^t ) is the posterior probability distribution of the acoustic model parameters described above, and can be obtained sequentially by setting an initial distribution appropriately. p (O _{t + 1} | θ _{t + 1} , O ^t ) is an output distribution of O _{t + 1} and is given by setting of an acoustic model such as HMM or GMM. Finally, p (θ _{t + 1} | θ _t , O ^t ) is the stochastic dynamics of the acoustic model parameter θ. Therefore, the recurrence formula of the formula (11) is composed of the initial distribution, the output distribution, and the stochastic dynamics.

Returning to FIG. 8, the current posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) obtained by the sequential learning unit 52 is temporarily stored in the posterior probability distribution storage unit 54. The current posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) is input to the model update unit 56.
In the model update unit 56, the previous posterior probability distribution p (θ _t | O ^t ) as the acoustic model in the acoustic model storage unit 58 is changed to the posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) of the current acoustic model parameter. Update as a new acoustic model (step S70).

Further, referring to FIG. 9, the obtained previous posterior probability distribution p (θ _t | O ^t ) is once stored in the acoustic model (distribution model) storage unit 58. The sequential learning unit 52 uses the previous posterior probability distribution p (θ _t | O ^t ) and the current feature amount sequence O ^{t + 1} to calculate the current posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ). In this way, the posterior probability distribution of the acoustic model parameters is updated sequentially.

[Introduction of Markov process]
Next, a method for simplifying the arithmetic processing of the equation (11) by assuming a Markov process will be described. p (O _{t + 1} | θ _{t + 1} , O ^t ) and p (θ _{t + 1} | θ _t , O ^t ) directly depend on the accumulated feature amount series. If these are to be estimated from all the accumulated feature quantity sequences, the accumulated data increases as time passes. Therefore, the estimation is very complicated and unrealistic. Therefore, assuming a Markov process, p (O _{t + 1} | θ _{t + 1} , O ^t ) and p (θ _{t + 1} | θ _t , O ^t ) are approximated as shown in Equation (13).
p (O _{t + 1} | θ _{t + 1} , O ^t ) ≈p (O _{t + 1} | θ _{t + 1} ),
p (θ _{t + 1} | θ _t , O ^t ) ≈p (θ _{t + 1} | θ _t ) (13)

By this approximation, the sequential learning unit 52 performs the posterior probability distribution p (θ _t | O ^t ) of the previous acoustic model parameter, the current output distribution p (O _{t + 1} | θ _{t + 1} ), and the current stochastic dynamics p (θ _{t + 1} | θ _t ) and posterior probability distribution p (θ _{t + 1} | O ^{t + 1} ) of the current acoustic model parameter. Specifically, it is approximated as the following formula (14).
p (θ _{t + 1} | O ^{t + 1} ) ∝p (O _{t + 1} | θ _{t + 1} ) ∫p (θ _{t + 1} | θ _t ) p (θ _t | O ^t ) dθ _t (14)
Here, A∝B indicates that A and B are proportional. A simple output distribution and stochastic dynamics can be set according to the equation (14). The sequential learning unit 52 in FIG. 8 calculates this equation (14).

[Consideration of mean vector of Gaussian distribution]
In the above discussion, the posterior probability distribution p (θ | O) of all acoustic model parameters θ such as the state transition probability a of the HMM, the mixture weight factor w of the GMM, the mean vector parameter μ of the Gaussian distribution, and the covariance matrix parameter Σ. The process was performed. In general, the parameter that determines the performance most in the acoustic model is the average vector parameter μ of the Gaussian distribution, and when the posterior probability distribution of other parameters is an estimation target, the number of parameters to be estimated by the distribution conversion function F is large. Therefore, the effect is not sufficient in small-scale data adaptation. Therefore, hereinafter, only the average vector parameter μ of the Gaussian distribution is focused, that is, the sequential learning unit 52 in FIG. 8 performs the calculation using the average vector parameter μ of the Gaussian distribution instead of the acoustic model parameter θ. Consider the time evolution of the computed posterior probability distribution p (μ | O). In other words, since only the average vector parameter μ of the Gaussian distribution is considered in the equation (14), the time evolution is calculated by the learning unit 52 sequentially as follows.
p (μ _{t + 1} | O ^{t + 1} ) p (O _{t + 1} | μ _{t + 1} ) ∫p (μ _{t + 1} | μ _t ) p (μ _t | O ^t ) dμ _t (15)
Equation (15) is independently given to the average vector parameter of each Gaussian distribution in the acoustic model. The index of each Gaussian distribution at that time is omitted in the text.

[Linear dynamics]
Next, let us consider deriving an analytical solution of the equation (15). Sequential learning is performed using this. There are various analytical solutions in equation (15), but consider the case where the stochastic dynamics is expressed linearly as the simplest analytical solution. That is, the following equation (16) can be assumed as the stochastic dynamics.
μ _{t + 1} = A _{t + 1} μ _t + ν _{t + 1} + ε _{t + 1} (16)

Here, ε _{t + 1} is a Gaussian noise having an average of 0 and a covariance matrix U. Equation (16) can be said to be that the linear transformation in equation (5) fluctuates stochastically. At this time, the specific distribution system of probability dynamics is given by the following equation (17).
p ([mu] _{t + 1} | [mu] _t ) = N ([mu] _{t + 1} | At _{+ 1 [} mu] _t + [nu] _{t + 1} , U) (17)

Here, N (x | m, S) in the equation (17) is a Gaussian distribution of an average parameter m and a covariance matrix parameter S having x as an argument. Further, an output distribution p (O _t | μ _t ) in which a group of feature amount series O _t = {o _{Nt + 1} ,..., O _{Nt + Nt + 1} } is output for an acoustic model expressed in normal HMM and GMM is as follows. It can represent with Formula (18).

Here, the zeta _n, a posteriori occupancy probability value of O _n assigned to a Gaussian distribution of the subject. The state transition probability a and the mixture weight factor w are ignored because they are not related to the estimation of p (μ | O). Although the latent variables of HMM and GMM are ignored, these can be dealt with by using the EM algorithm (expected value maximization algorithm). Actually, Expression (18) is expressed in the form of an auxiliary function in the EM algorithm.

Finally, assuming that the posterior probability distribution p (μ _t | O ^t ) of the average vector parameter of the Gaussian distribution in the acoustic model is expressed by a Gaussian distribution, the average vector parameter of the posterior probability distribution p (μ _t | O ^t ) was a mu ^ _t, when the covariance matrix parameters posterior distribution p (μ _t │O ^_t) is represented by Q ^ _t function form can be expressed by the following equation (19).
p (μ _t | O ^t ) = N (μ _t | μ ^ _t , Q ^ _t ) (19)

Therefore, an analytical solution represented by the following equation (20) can be derived by substituting equations (17), (18), and (19) into equation (15).
p (μ _{t + 1} | O ^{t + 1} ) = N (μ _{t + 1} | μ ^ _{t + 1} , Q ^ _{t + 1} ) (20)
here,
_{Q ^ t + 1 = ((} U + A t + 1 Q ^ t A t + 1 ') -1 + ζ t + 1 Σ -1) -1
(21)
K ^ _{t + 1} = Q ^ _{t + 1} ζ _{t + 1} Σ- ¹ (22)
μ ^ _{t + 1} = A _{t + 1} μ ^ _t + ν _{t + 1}
+ K ^ _{t + 1} (M _{t + 1} / ζ _{t + 1} −A _{t + 1} μ ^ _t −ν _{t + 1} ) (23)

However, μ _{^ t + 1} this time of the posterior probability distribution p the ^{_{(μ t + 1 │O t +}} 1) is the mean vector parameter when expressed by a Gaussian distribution, _{Q ^ t + 1} this time of the posterior probability distribution p ^{_{(μ t + 1 │O t +}} 1) Is a covariance matrix parameter when K is expressed by a Gaussian distribution, K _{t + 1} is a Kalman gain, and A _{t + 1} , ν _{t + 1} , and U are each a linear expression of the above stochastic dynamics in the acoustic model parameters. Is a covariance matrix of the coefficient matrix, coefficient vector, and Gaussian noise, Σ is the covariance matrix in the initial acoustic model parameters, and A _{t + 1} ′ represents the transpose of the matrix At _{+ 1} . ζ _{t + 1} is the sum of the current posterior occupancy probability values, M _{t + 1} is the product sum of ζ and the feature value at each time point, and ζ _t and M _t are sufficient statistics of the average vector parameter of the Gaussian distribution. And is defined as the following formula (24).

ζ _t and M _t can be efficiently obtained by an alignment method such as the Forward-backward algorithm, Viterbi algorithm, or kmeans method. All posterior probability distributions can be updated by updating the equations (21) to (23) for all Gaussian distributions in the acoustic model.

In this way, the distribution parameter ω _{t + 1} of the current posterior probability distribution p (μ _{t + 1} | O ^{t + 1} ) becomes Q ^ _{t + 1} , μ ^ _{t + 1} (see the parentheses in the acoustic model parameter storage unit 58 in FIG. 9), and the equation ( 21) (22) (23). Note that K ^ _{t + 1} in equation (22) is introduced for ease of expression of the equation. In actual calculation, equations (22) and (23) may be performed simultaneously. In that case, it is not necessary to obtain K ^ _{t + 1} .

That is, the recurrence formula of the posterior probability distribution p (μ | O) of the acoustic model parameter can be obtained by the recurrence formulas (21), (22), and (23) of the parameters (Q ^, μ ^). This is similar to the Kalman filter solution in linear dynamic systems. However, the solution of the Kalman filter has become the update for each voice analysis frame as o _n → o _{n + 1.} On the other hand, in the present invention, the difference is that updating is performed for each frame as O _t → O _{t + 1} . Therefore, the parameter Q ^, mu ^, not the feature amount o _n of one frame are represented by the statistic. Therefore, this is called a macroscopic linear dynamic system.

FIG. 11 shows a specific configuration example of the sequential learning unit 52 when Q ^ and μ ^ are used. The sequential learning unit 52 includes a Q ^ update unit 520, a K ^ update unit 522, a μ ^ update unit 524, and a posterior probability calculation unit 526.
The Q ^ update unit 520 calculates the equation (21), the K ^ update unit 522 calculates the equation (22), the μ ^ update unit 524 calculates the equation (23), and the posterior probability calculation unit 526. Then, the equation (20) is calculated.

Therefore, in order to obtain Q ^ and μ ^, it is necessary to set four of linear transformation parameters W _{t + 1} = {ν _{t + 1} , A _{t + 1} }, system noise U _{t + 1} , initial parameters Q ^ ₀ , and μ ^ _0. is there. Here, Q ^ ₀ is given from the covariance matrix parameter of the initial acoustic model, and μ ^ ₀ is given from the average vector parameter of the initial acoustic model.

Among these, the linear transformation parameter W = {ν _{t + 1} , A _{t + 1} } is estimated using at least one feature amount sequence among the feature amount sequences O ^t accumulated up to this time. As an example of a well-known method, it is efficiently obtained by repeated calculation using the above-described EM algorithm or MLLR algorithm. Further, by sharing the same conversion parameter among a plurality of Gaussian distributions, it is possible to reduce the number of parameters to be estimated.

Similarly to the linear transformation parameter W, the system noise U can be obtained by learning. Alternatively, all matrix components can be given a priori from a feature series or other data. The simplest method is the system noise U (u ⁰⁾ leave the ^-1 sigma, as the covariance matrix of system noise and is proportional to the covariance matrix of the output distribution, a parameter given to u ⁰ in advance To do. That is, only one parameter is introduced. The system noise U and the linear conversion parameter W are conversion parameters in the distribution conversion function F of the equation (8).

At this time, the update formula is expressed by the following formulas (25), (26), and (27), the Q ^ update unit 520 calculates the formula (25), and the K ^ update unit 522 calculates the formula (26). Then, the μ ^ update unit 524 calculates the equation (27).
^{_{Q ^ t + 1 = ((}} (u 0) -1 Σ + A t + 1 Q ^ t A t + 1 ') -1 + ζ t + 1 Σ -1) -1 (25)
K ^ _{t + 1} = Q ^ _{t + 1} ζ _{t + 1} Σ- ¹ (26)
_{μ ^ t + 1 = A t} + 1 μ ^ t + ν t + 1 + K ^ t + 1 (M t + 1 / ζ t + 1 -A t + 1 μ ^ t -ν t + 1) (27)
As described above, the sequential adaptation method based on the distribution transformation controlled by the parameter u ⁰ can be realized.

[Translation adaptation]
By paying attention only to the translation ν _{t + 1} of the mean vector μ _t in the equation (16) of the linear dynamics, it is possible to reduce the number of parameters to be estimated and realize adaptation with a smaller amount of data. At this time, assuming that the matrix A _{t + 1} in the equations (25), (26), and (27) is the unit matrix I, that is, A _{t + 1} = I, Q ^, K ^, and μ ^ are the following expressions (28) ( 29) Calculated in (30).

^{_{Q ^ t + 1 = ((}} (u 0) -1 Σ + Q ^ t ') -1 + ζ t + 1 Σ -1) -1 (28)
K ^ _{t + 1} = Q ^ _{t + 1} ζ _{t + 1} Σ- ¹ (29)
μ ^ _{t + 1} = μ ^ _t + νt _{+ 1} + Q ^ _{t + 1} ζt _{+ 1} Σ− ¹ (Mt _{+ 1} / ζt _{+ 1−} μ ^ _t− νt _{+ 1} ) (30)
In this case, the Q ^ update unit 520 calculates the equation (28), the K ^ update unit 522 calculates the equation (29), and the μ ^ update unit 524 calculates the equation (30).
CJLeggetter and PCWoodland, Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models.Computer Speech and Language, Vol.9, pp.171-185, 1995. CJLeggetter and PCWoodland, Maximum.Flexible speaker using maximum likehood linear regression In Proc ARPA Spoken Language Technology Work-shop, pp.104-109,1995. Satoshi Watanabe, Atsushi Nakamura, Sequential model adaptation based on macroscopic time evolution system of probability distribution. Proceedings of the Autumn Acoustics Society, 2-2-10, pp. 71-72, 2006. Yuji Watanabe, Atsushi Nakamura, Consideration of the relationship between model adaptation based on the macroscopic time evolution system of probability distribution and conventional adaptation. Proceedings of the Autumn Acoustics Society 2-3-12, 2007. JP 2008-64849 A

In the online sequential adaptation task, model update in a very short time of about 1 second is necessary for real-time processing. However, the update of the adaptive method based on the distribution transformation is expressed by the operations of the matrix Q ^ _t and Σ, as can be seen from the equations (28) to (30), and all the product, sum and inverse matrix operations of this matrix are performed. It is necessary to carry out a Gaussian distribution, and the calculation cost is very high, and real-time processing is difficult. For example, a normal acoustic model includes tens of thousands of 39-dimensional Gaussian distributions, but executing equations (28) to (30) on this is a 39 × 39 matrix A, Q ^ _t or Σ (however, , .SIGMA. Usually uses a diagonal matrix), and it is necessary to perform the product, sum, and inverse matrix for the number of Gaussian distributions (tens of thousands of times).

Also, since equations (28) to (30) include calculation of the inverse matrix, the calculation becomes unstable depending on the voice data, and the inverse matrix cannot be obtained.
Further, since the distribution parameter Q ^ _t is necessary for updating the model, it is necessary to store them in the acoustic model parameter storage unit 58. However, Q ^ _t is a total covariance matrix (where the off-diagonal component is not 0) (however, a symmetric matrix), and there are as many Gaussian distributions in the acoustic model, and thus a large amount of memory is consumed. For example, an acoustic model represented by tens of thousands of 39-dimensional Gaussian distributions is about several megabytes, whereas only Q ^ _t consumes 10 times more memory (tens of megabytes) than the acoustic model.

  As described above, the adaptive method based on the distribution transformation has a large amount of calculation, is unstable in calculation, and consumes a lot of memory. Therefore, it has been difficult to realize an online sequential adaptation task that uses them for sequential update in a very short time of about 1 second.

  In the present invention, an acoustic model creation device that reduces the amount of calculation and the amount of memory and improves the stability of calculation compared to the prior art, a speech recognition device using the device, these methods, these programs, and these A recording medium is provided.

  The acoustic model creation device of the present invention includes a feature extraction unit, a sequential learning unit, and a model update unit. The feature extraction unit extracts a feature amount series of the current adaptation audio data. The sequential learning unit expresses the posterior probability distribution of the average vector parameter of the Gaussian distribution in the acoustic model as a Gaussian distribution, the scaling factor for the average vector parameter of the posterior probability distribution, the covariance matrix parameter of the posterior probability distribution, The posterior probability distribution of the average vector parameter of the Gaussian distribution in the acoustic model obtained last time and the current time, taking into account the feature quantity series accumulated up to the previous time based on the covariance matrix in the initial acoustic model parameters The posterior probability distribution average vector parameter and the posterior distribution when the posterior probability distribution of the average vector parameter of the Gaussian distribution in the current acoustic model is expressed as a Gaussian distribution using a part of the feature amount series accumulated up to This acoustic model is calculated by calculating the scaling factor of the covariance matrix parameter of the probability distribution. Determining the posterior probability distribution of the mean vector parameters of the Gaussian distribution. The model update unit converts the posterior probability distribution of the current acoustic model parameter into a new acoustic model parameter and updates it.

In the present invention, with respect to the posterior probability distribution p (μ _t | O ^t ) of an average vector parameter μ _t of a certain Gaussian distribution in the acoustic model, the covariance matrix parameter Q ^ _t is used as the Gauss of the target acoustic model. Using the covariance matrix Σ of the distribution and the reciprocal (r ^ _t ) ⁻¹ of the scaling factor r ^ _t , which is a scalar variable newly introduced in the present invention, it is expressed by the following equation. .
^{_{Q ^ t = (r ^ t}} ) -1 Σ (31)
Thereby, since Formulas (28) to (30) can be rewritten into a scalar calculation, the calculation amount can be reduced and the stability can be ensured. Further, since the update parameter to be stored is changed from the symmetric matrix Q ^ _t to r ^ _t , the memory capacity in the acoustic model storage unit can be reduced.

  The best mode for carrying out the invention will be described below. In addition, the same number is attached | subjected to the process which performs the same part and the process which has the same function, and duplication description is abbreviate | omitted.

First, the symbol is defined again.
μ _t Average vector parameter of Gaussian distribution in previous acoustic model Σ _t Covariance matrix parameter of Gaussian distribution in previous acoustic model p (μ _t | O ^t ) Average vector parameter of Gaussian distribution in previous acoustic model μ I mean vector parameter in mean vector parameter mu _t posterior probability distribution p of the posterior distribution probability mu ^ _t Gaussian distribution in the acoustic model of _t a (μ _t │O ^_t) is expressed by the Gaussian distribution, or, p (mu _t | O ^t ) average vector parameter Q ^ _t covariance matrix parameter when the posterior probability distribution p (μ _t | O ^t ) of the average vector parameter μ _t of the Gaussian distribution in the acoustic model is expressed by a Gaussian distribution, or p (μ ^_t │O _t) of the covariance matrix parameters r _{^ t} mean vector parameters of the Gaussian distribution in the acoustic model mu _t posterior probability distribution p (mu scaling factor _{^t │O t)} for the covariance matrix parameters Q _{^ t} when expressed in Gaussian distribution or the scaling factor for the covariance matrix parameters Q _{^ t} of p (μ _{^t │O t)}

  FIG. 12 shows a functional configuration example of the acoustic model creation apparatus according to the first embodiment. The flow of processing will be described with reference to FIG. 10. FIG. 13 shows a procedure for converting acoustic model parameters when the sequential adaptation method is used. FIG. 14 shows a functional configuration example of the sequential learning unit. As illustrated in FIG. 12, the acoustic model creation device 148 includes a feature extraction unit 4, a feature amount storage unit 5, and a model adaptation unit 150. The model adaptation unit 150 includes a sequential learning unit 152 and a posterior probability storage. Unit 154 and model update unit 156. The sequential learning unit 152 includes an r ^ update unit 1522, a μ ^ update unit 1524, and a posterior probability calculation unit 1526.

First, the average vector parameter μ ^ _t and scaling factor r ^ _t (r ^ _t will be described later) of the previous posterior probability distribution are read by the model adaptation unit 150 (step S60). Then, the adaptation audio data 20 is read (step S62), and the adaptation audio data is input to the feature extraction unit 4 and converted into the feature amount series O _{t + 1} (step S64). The converted feature quantity sequence O _{t + 1} is temporarily stored in the feature quantity storage unit 5 and is sequentially input to the learning unit 152.

As the processing of the sequential learning unit 152, first, (i) the posterior probability distribution p (μ _t | O ^t ) of the average vector parameter μ _t of the Gaussian distribution in the acoustic model is represented by the posterior probability distribution p (μ _t | O ^t ) average vector parameter μ ^ _t , scaling factor r ^ _t for covariance matrix parameter Q ^ _t of posterior probability distribution p (μ _t | O ^t ), and covariance matrix Σ in the initial acoustic model parameters Based on the Gaussian distribution. Then, (ii) the posterior probability distribution p (μ _t | O ^t ) of the mean vector parameter μ _t of the Gaussian distribution in the acoustic model obtained in the previous time, in which the accumulated feature amount sequence O ^t up to the previous time is added. iii) A part of the feature amount series O ^t accumulated so far is used. (Iv) By calculating the mean vector parameter μ ^ _{t + 1} and the covariance matrix scaling factor r ^ _{t + 1} when the a posteriori probability distribution of the average vector parameter of the Gaussian distribution in the current acoustic model is expressed by the Gaussian distribution, The posterior probability distribution p (μ _{t + 1} | O ^{t + 1} ) of the average vector parameter of the Gaussian distribution in the current acoustic model is obtained.

Details will be described below. Further, in this embodiment, in order to focus on improving the calculation amount, stability, and memory amount, instead of linear regression adaptation, the translation adaptation with a small number of parameters (explained in the paragraph [Parallel adaptation]) is used. Proceed with discussions. Hereinafter, the description will be divided into (i) to (iv).
Regarding the equations (28) to (30), equations obtained by substituting K ^ _{t + 1} shown in the equation (29) into the equation (30) are shown below.
^{_{Q ^ t + 1 = ((}} (u 0) -1 Σ + Q ^ t) -1 + ζ t + 1 Σ -1) -1 (32)
μ ^ _{t + 1} = μ ^ _t + νt _{+ 1} + Q ^ _{t + 1} ζt _{+ 1} Σ− ¹ (Mt _{+ 1} / ζt _{+ 1−} μ ^ _t− νt _{+ 1} ) (33)

Then, in order to reduce the number of distribution parameters, as shown in the equation (31), co-Gaussian distribution of a covariance matrix Q ^ _t the acoustic model of the posterior distribution to the mean vector mu _t Gaussian distribution of acoustic models This is expressed by multiplying the variance matrix Σ by the inverse of the scaling factor r ^ _t or r ^ _t (r ^ _t ) ^-1 . As a precaution, equation (31) is shown below. The scaling factor r ^ _t is a parameter represented by a real number (scalar).
^{_{Q ^ t = (r ^ t}} ) -1 Σ (31)
Substituting this equation (31) into equation (32), the update equations for Q ^ _{t + 1} can be expressed as follows.
Q ^ _{t + 1} = ((([mu] ⁰ ) ^{-1 [} Sigma] + (r [tau _] ) ^{-1 [} Sigma]) ^- 1+ [zeta] _{t + 1 [} Sigma] ^-1 ) ^-1.
= (((Μ ⁰ ) ⁻¹ + (r ^ _t ) ⁻¹ ) ⁻¹ + ζ _{t + 1} ) ⁻¹ Σ
(34)

Since the right side of the expression Q ^ _{t + 1} = (r ^ _{t + 1} ) ^-1 Σ obtained by modifying the expression (31) is equal to the right side of the expression (34), the following expression (35) is established.
_{r ^ t + 1 = ((} μ 0) -1 + (r ^ t) -1) -1 + ζ t + 1 (35)
That is, the update formula of Q ^ _{t + 1} shown in Expression (32) can be rewritten to r ^ _{t + 1} shown in Expression (35).

On the other hand, considering the update equation for μ ^ _{t + 1} , substituting equation (31) into equation (33) yields the following equation.

Here, as described above, μ ⁰ is a predetermined constant, r ^ _t is the previous scaling factor, ζ _{t + 1} is the sum of the current posterior occupation probability values, and M _{t + 1} is Ν _{t + 1} is a coefficient vector when linearly expressing the average stochastic dynamics in the current acoustic model parameter. Also, the reason for using the reciprocal (r ^ _t ) ^-1 of the scaling factor r ^ _t is that if r ^ _t is used as it is, the left side of equation (35) becomes (r ^ _{t + 1} ) ^-1. It is an expression problem. Either can be used for implementation. Further, arbitrary real values are given for the initial values ζ ₀ , M ₀ , r ₀ . Moreover, the following formula | equation (37) is calculated | required by substituting a formula (31) to Q ^ _t of a formula (20).
p (μ _{t + 1} | O ^{t + 1} ) = N (μ _{t + 1} | μ ^ _{t + 1} , (r ^ _{t + 1} ) ⁻¹ Σ)
(37)

That is, as described in the above (i), from the equation (37), the posterior probability distribution p (μ _t | O ^t ) of the average vector parameter μ _t of the Gaussian distribution in the acoustic model is expressed as the posterior probability distribution p. (μ ^_t │O _t) mean vector parameters mu _{^ t,} the posterior distribution p (μ ^_t │O _t) covariance matrix parameters Q _^ scaling factor for _t r _{^ t,} covariance in the initial acoustic model parameters It will be understood that it is represented by a Gaussian distribution represented by a matrix Σ.

Further, (ii) a posteriori probability distribution p (μ _t | O ^t ) of the average vector parameter μ _t of the Gaussian distribution in the acoustic model obtained in the previous time, which is added with the feature amount series O ^t accumulated up to the previous time. The use will be described. The equations (35) and (36) are used to calculate the equation (37). As is clear from the equations (35) and (36), the previous average vector parameter μ ^ _t and the previous scaling factor are used. r ^ _t must be used. Further, the posterior distribution probability p (μ _t | O ^t ) of the acoustic model parameter obtained last time from the equation (37) is
p (μ _t | O ^t ) = N (μ _t | μ ^ _t , (r ^ _t ) ⁻¹ Σ) (37 ′)
It is represented by Therefore, using the previous average vector parameter μ ^ _t and the previous scaling factor r ^ _t means using the posterior distribution probability p (μ _t | O ^t ).

Also, (iii) using a part of the feature amount series O ^t accumulated up to this time will be described. The current coefficient vector ν _{t + 1} is estimated using the EM algorithm or the MLLR algorithm as described above. The estimation uses a part of O ^t = O ₁ , O ₂ ,..., O _t . Then, the a posteriori probability distribution p (μ _{t + 1} | O ^{t + 1} ) of the current acoustic model parameter is obtained by Expression (37).

In addition, (iv) expressing the posterior probability distribution of the average vector parameter of the Gaussian distribution in the current acoustic model as a Gaussian distribution will be described. The posterior probability distribution p (μ _t of the average vector parameter of the Gaussian distribution in the acoustic model. This is because the above expression (19) is expressed by assuming that | O ^t ) is expressed by a Gaussian distribution.

Referring to FIG. 14, the r ^ update unit 1522 calculates r ^ by calculating the equation (35), and the μ ^ update unit 1524 updates μ ^ by calculating the equation (36). The posterior probability calculation unit 1526 calculates the formula (37) to obtain the posterior probability distribution p (μ _{t + 1} | O ^{t + 1} ) of the current acoustic model parameter. The description of the model update unit 156 will be described in the second embodiment. Further, the modified example will be described. The equation (35) is a suitable example. If the equation is close to the equation (35), the current scaling factor r ^ _{t + 1} is the previous scaling factor r ^ _t , It can be obtained from the sum ζ _{t + 1} of the posterior occupation probability values. Similarly, the equation (36) is a suitable example. If the equation is close to the equation (36), the current average vector parameter μ ^ _{t + 1} is the previous average vector parameter μ ^ _t and the current acoustic model. Coefficient vector ν _{t + 1} when linearly expressing the average probabilistic dynamics in the parameter, the sum ζ _{t + 1} of the posterior occupation probability value this time, the product sum M _{t + 1} of ζ and the feature quantity at each time point, the current scaling factor r ^ _{t + 1} .

Then, the model update unit 56 converts the posterior probability distribution p (μ _{t + 1} | O ^{t + 1} ) in the current acoustic model parameter into a new acoustic model parameter and updates it (step S70).

Next, the effect of the invention of this embodiment will be described. As can be seen from the equations (35) and (36), the covariance matrix Q ^ _t of the distribution parameter is replaced with r ^ _t and Σ by using the equation (31), and Σ is canceled out. It can be seen that the matrix representation in the equation has been removed. Further, comparing the update formulas (35) and (36) of the first embodiment with the conventional update formulas (32) and (33), the update formulas (35) and (36) of the first embodiment are calculated by matrix calculation (product, (Sum, Inverse matrix) is not required, so calculation is fast and stability is ensured.
Further, by using the equations (35) and (36), the covariance matrix Q ^ _t of the distribution parameter and the average vector parameter μ ^ _t are recorded in the sequential adaptation as described in the acoustic model storage unit 58 in FIG. Instead, the amount of memory can be significantly reduced by recording the scaling factor r ^ _t and the average vector μ ^ _t as described in the acoustic model storage unit 158 in FIGS.

  With the above method, the posterior probability distribution p (μ | O) of the acoustic model parameters, that is, the acoustic model is obtained. In the second embodiment, a speech recognition process using the obtained acoustic model, that is, a process of calculating an acoustic score will be described. FIG. 15 shows an example of the functional configuration of the speech recognition apparatus of this embodiment, and FIG. 16 shows the main processing flow of speech recognition.

  Adaptation speech data having an acoustic feature similar to the acoustic feature amount of the recognition speech data is input to the acoustic model creation device 148 described in the first embodiment. Then, the acoustic model in the acoustic model storage unit 158 is updated as described above (step S80). The recognition speech data is divided into frames and input to the feature extraction unit 4 as recognition speech data x and converted into a feature amount series O. The feature amount series O is input to the word string search unit 6 (step S82).

The word string search unit 6 calculates an acoustic score of each Gaussian distribution for the feature amount series O using the acoustic model in the acoustic model storage unit 8 as necessary. For this acoustic score calculation, for example, the following equation (40) is calculated.
_{∫p (x τ | μ t)} p (μ t | O t) dμ t (40)
Here, p (x _τ | μ _t ) is the output distribution of the acoustic model. parameters other than μ _t is omitted here. Therefore, p (μ _t | O ^t ) may be examined. Regarding the acoustic score calculation for a plurality of frames by the word string search unit 6, dynamic programming (DP: Dynamic Programming matching) may be performed based on the equation (40). The word string that maximizes the acoustic score is output as a recognized word string (step S84). In this case, the model update in step S80 updates the posterior probability distribution p (μ _τ | O ^t ) as an acoustic model (step S80a). The integral of the equation (40) can be solved numerically, but there are two types of analytical solutions as follows.

従って、音響モデル作成装置６０による前記ステップＳ８０におけるモデル更新として、前記式（３６）で求まるμ＾_ｔを出力分布ｐ(ｘ_τ｜μ_ｔ)の平均ベクトルパラメータμ_ｔにそのまま代入（Plug-in）して音響モデルパラメータを更新する（ステップＳ８０ｂ）。このようにすればスコア計算を、以下の式（４２）で行うことが出来る。

つまり、平均μ＾_ｔ、共分散行列Σのガウス分布で表現する。これをPlug-in法と呼ぶ。また、その他のパラメータ状態遷移確率ａ、混合重み因子ｗ、共分散行列Σ、はそのまま適用する。ステップＳ８０ｂの後は、破線矢印で示すように、ステップＳ８２に移る。 [Plug-in method]
In the Plug-in method, the integral is not handled properly, but the posterior probability maximization (MAP) value (the right side of the following equation (41)) of p (μ _t | O ^t ) is expressed by the equation (36). Utilize that μ ^ _t . That is, the following expression (41) is obtained.

Therefore, as a model updating in the step S80 by the acoustic model creating apparatus 60, the equation obtained in (36) μ ^ _t the output distribution _p (x τ | μ _t) as it is assigned to the mean vector parameter mu _t of (Plug-in ) To update the acoustic model parameters (step S80b). In this way, the score can be calculated by the following formula (42).

That is, it is expressed by a Gaussian distribution of mean μ ^ _t and covariance matrix Σ. This is called the plug-in method. The other parameter state transition probability a, the mixture weight factor w, and the covariance matrix Σ are applied as they are. After step S80b, the process moves to step S82 as indicated by the broken line arrow.

つまり、周辺化法を利用する場合はステップＳ８０のモデル更新において、平均ベクトルパラメータμをμ＾_ｔと置き換える（ステップＳ８０ｂ）と共に、共分散行列パラメータΣ→Σ＋（ｒ＾_ｔ）^−１Σと置き換えて（ステップＳ８０ｃ）、音響モデルパラメータを更新する。また、その他のパラメータ、つまり、状態遷移確率ａ、混合重み因子ｗ、はそのまま適用する。ステップＳ８０ｃのあとは、ステップＳ８２に移る。 [Marginalization method]
Unlike the plug-in method, the marginalization method is an analytical solution of the integral. How to solve this integral corresponds to the peripheral of the mean vector parameter mu _t. The marginalization method considers the variance of the posterior probability distribution p (μ _t | O ^t ) of the average vector parameter as compared to the plug-in method. If it does in this way, the score calculation by integral calculation can be represented by the following formula | equation (43).

That is, when the marginalization method is used, the average vector parameter μ is replaced with μ ^ _t (step S80b) and the covariance matrix parameter Σ → Σ + (r ^ _t ) ⁻¹ Σ in the model update in step S80. (Step S80c), the acoustic model parameters are updated. Other parameters, that is, the state transition probability a and the mixture weight factor w are applied as they are. After step S80c, the process proceeds to step S82.

Further, as shown in FIG. 14, the model update unit 156 functions as a parameter conversion unit indicated by a broken line. When using the Plug-in method, the average vector parameter is updated by substituting μ → μ ^ _t, and when using the marginalization method, μ → μ ^ _t , Σ → Σ + (r ^ _t ) ⁻¹ Σ And the mean vector parameter and covariance matrix parameter are updated. By doing in this way, the acoustic score by the sequential adaptation method based on distribution conversion is computable.

[Experimental result]
Using ASJ (acoustic societies) reading speech database for 100 hours, we built an unspecified speaker acoustic model with a total of 2000 triphone HMM states and 16 mixed states per HMM state. A sequential adaptation experiment was conducted. The feature quantity uses a triangle with a vocabulary size of 700,000 words as the 12-dimensional MFCC (mel frequency pepstrum coefficient), the energy of the frame, the inter-frame difference Δ of the MFCC, and the inter-frame difference delta ΔΔ of the difference MFCC. A large vocabulary continuous speech recognition experiment was conducted. The speech recognition rate in the case of normal speech recognition without sequential adaptation was 81.3%.

Here, in the conventional sequential adaptation using the covariance matrix Q ^ _t as the distribution parameter, the recognition rate is greatly improved to 88.5%. However, the update is performed sequentially with one utterance (about 1 second). In the online sequential adaptation task, the conventional method cannot perform real-time processing (about twice the real time), and the memory consumption used for Q ^ _t is 27 megabytes.

On the other hand, in the present invention using the scaling factor r ^ _t as the distribution parameter, the recognition rate is 88.5%, and real-time processing (about 1 time of real time) is realized while maintaining the same performance as the conventional method. . The memory consumption used for r ^ _t was 1.3 megabytes, and the memory could be reduced by about 20 times compared to the case where Q ^ _t was used.

In addition to the above-described embodiments, the acoustic model creation device according to the present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the acoustic model creation device is not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the device that executes the processing. Good.
When the processing in the acoustic model creation device of the present invention is realized by a computer, the processing contents of the functions that the acoustic model creation device should have are described by a program. Then, by executing this program on a computer, the processing function in the acoustic model creation device is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like is used as an optical disc, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable). ) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable Programmable-Read Only Memory), etc. can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Further, the above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the acoustic model creation apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

The figure which showed the function structural example of the conventional speech recognition apparatus. The figure which showed the processing flow of the conventional speech recognition apparatus. The figure which showed the function structural example of the conventional acoustic model production apparatus. The figure which showed the processing flow of the conventional acoustic model production apparatus. The figure which showed the function structural example of the acoustic model creation apparatus at the time of using a linear regression matrix. The block diagram which showed the flow of the main processes of the acoustic model production apparatus shown in FIG. The figure which showed the procedure in which the acoustic model parameter at the time of using a sequential adaptation method is converted. The figure which showed the function structural example of the acoustic model creation apparatus at the time of applying a sequential response. The figure which showed the order which adapts the parameter of posterior probability distribution sequentially. The block diagram which showed the flow of the main processes of the acoustic model production apparatus shown in FIG. The figure which showed the function structural example of the conventional sequential learning part 52. FIG. The figure which showed the function structural example of the acoustic model production apparatus of a present Example. The figure which showed the order which adapts the parameter of the posterior probability distribution of a present Example sequentially. The figure which showed the function structural examples, such as the sequential learning part of a present Example. The figure which showed the function structural example of the speech recognition apparatus of a present Example. The figure which showed the processing flow of the speech recognition apparatus of a present Example.

Claims (10)

A feature extraction unit that extracts a feature amount sequence of the adaptive audio data;
When the posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model is expressed by the Gaussian distribution, the mean vector parameter of the posterior probability distribution, the scaling factor for the covariance matrix parameter of the posterior probability distribution, and the initial acoustic model parameter The posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model obtained last time and the feature accumulated up to this time, taking into account the feature quantity series accumulated up to the previous time based on the covariance matrix Covariance of the posterior probability distribution's average vector parameter and the posterior probability distribution when the posterior probability distribution of the average vector parameter of the Gaussian distribution in this acoustic model is expressed as a Gaussian distribution using a part of the quantity series Gaussian distribution in this acoustic model by calculating the scaling factor of the matrix parameter A sequential learning section for determining the posterior probability distribution of the mean vector parameter,
An acoustic model creation device, comprising: a model updating unit that converts the posterior probability distribution of the current acoustic model parameter into a new acoustic model parameter and updates it. The acoustic model creation device according to claim 1,
The sequential learning unit calculates the current scaling factor r ^ _{t + 1} for the covariance vector parameter of the posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model, the previous scaling factor r ^ _t , and the current posterior occupation probability value. From the sum ζ _{t + 1} ,
The present average vector parameter μ ^ _{t + 1} of the posterior probability distribution of the average vector parameter of the Gaussian distribution in the acoustic model is linearly expressed as the previous average vector parameter μ ^ _t and the average stochastic dynamics in the current acoustic model parameter. Acoustics characterized by being obtained from the coefficient vector ν _{t + 1 at} the time, the sum ζ _{t + 1 of} the current posterior occupancy probability value, the product sum M _{t + 1} of ζ and the feature value at each time point, and the current scaling factor r ^ _{t + 1.} Model creation device. The acoustic model creation device according to claim 2,
The sequential learning unit obtains a scaling factor r ^ _{t + 1} of the current acoustic model parameter and an average vector parameter μ ^ _{t + 1} of the current acoustic model parameter by the following equation:

However, μ ⁰ is a predetermined constant, r ^ _t is the previous scaling factor, ζ _{t + 1} is the sum of the posterior occupation probability values of this time, and M _{t + 1} is ζ and the feature value at each time point of the current time. And ν _{t + 1} is a coefficient vector obtained by linearly expressing the average stochastic dynamics in the current acoustic model parameter. A model for recognition in which an acoustic model adapted to voice data for adaptation having acoustic characteristics of voice data for recognition is created by the acoustic model creation device according to any one of claims 1 to 3, and the acoustic model is updated. Update section,
A speech recognition apparatus comprising: a recognition unit configured to perform speech recognition on input speech data having the acoustic feature using the updated acoustic model. A feature extraction process for extracting a feature amount sequence of adaptive audio data;
When the posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model is expressed by the Gaussian distribution, the mean vector parameter of the posterior probability distribution, the scaling factor for the covariance matrix parameter of the posterior probability distribution, and the initial acoustic model parameter The posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model obtained last time and the feature accumulated up to this time, taking into account the feature quantity series accumulated up to the previous time based on the covariance matrix Covariance of the posterior probability distribution's average vector parameter and the posterior probability distribution when the posterior probability distribution of the average vector parameter of the Gaussian distribution in this acoustic model is expressed as a Gaussian distribution using a part of the quantity series Gaussian distribution in this acoustic model by calculating the scaling factor of the matrix parameter A sequential learning process for obtaining the posterior distribution of the mean vector parameter,
A model updating step of converting the posterior probability distribution of the current acoustic model parameter into a new acoustic model parameter and updating it. The acoustic model creation method according to claim 5,
In the sequential learning process, the current scaling factor r ^ _{t + 1} for the covariance vector parameter of the posterior probability distribution of the mean vector parameter of the Gaussian distribution in the acoustic model, the previous scaling factor r ^ _t , and the current posterior occupation probability value From the sum ζ _{t + 1} ,
The present average vector parameter μ ^ _{t + 1} of the posterior probability distribution of the average vector parameter of the Gaussian distribution in the acoustic model is linearly expressed as the previous average vector parameter μ ^ _t and the average stochastic dynamics in the current acoustic model parameter. Acoustics characterized by being obtained from the coefficient vector ν _{t + 1 at} the time, the sum ζ _{t + 1 of} the current posterior occupancy probability value, the product sum M _{t + 1} of ζ and the feature value at each time point, and the current scaling factor r ^ _{t + 1.} Model creation method. The acoustic model creation method according to claim 6,
In the sequential learning process, a scaling factor r ^ _{t + 1} of the current acoustic model parameter and an average vector parameter μ ^ _{t + 1} of the current acoustic model parameter are obtained by the following equations:

However, μ ⁰ is a predetermined constant, r ^ _t is the previous scaling factor, ζ _{t + 1} is the sum of the posterior occupation probability values of this time, and M _{t + 1} is ζ and the feature value at each time point of the current time. And ν _{t + 1} is a coefficient vector when linearly expressing the average stochastic dynamics in the current acoustic model parameters. A recognition model for updating an acoustic model by creating an acoustic model adapted to the adaptation speech data having acoustic characteristics of the recognition speech data by the acoustic model creation method according to any one of claims 5 to 7. Update process,
A speech recognition method comprising: a speech recognition process for performing speech recognition on input speech data having the acoustic features using the updated acoustic model.   The program which operates a computer as the acoustic model production apparatus in any one of Claims 1-3, or the speech recognition apparatus of Claim 4.   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images