JPH1185187A - Acoustic model generating device and speech recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the acoustic model generating device and the speech recog nition device in which the characteristics of voice data are more precisely reflected, the power of expression of a general speaker acoustic model is im proved compared with a conventional example and voice is recognized with a higher voice recognition rate. SOLUTION: An initial hidden Markov model(HMM) generating section 21 generates an initial HMM by a prescribed learning algorithm based on the feature parameter of the voice data. An HMM reconstituting section 22 reconstitutes an initial HMM by adding the components having an HMM Gauss mixed distribution based on the tendency of the frame errors which are discrimination errors of the frame unit caused by an initial HMM concerning the voice data and generates the reconstituted HMM. A relearning section 23 relearns the reconstituted HMM, which is relearned by a prescribed learning algorithm, based on the feature parameter of the voice data and generates the acoustic model which is the relearned HMM.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an acoustic model generating apparatus for generating an acoustic model such as a Hidden Markov Model (hereinafter, referred to as HMM), and an uttered speech sentence input using the generated acoustic model. The present invention relates to a speech recognition device for recognizing speech based on a speech signal.

[0002]

2. Description of the Related Art In an unspecified speaker's speech recognition apparatus, HMM as an acoustic model is required to have both precision and robustness in order to improve recognition performance. Improvement of precision is a problem of how to faithfully model an acoustic phenomenon for each acoustic unit (for example, phoneme) based on an actual speech sample. However, in practice, if the precision is sought too much due to a shortage of usable audio samples, modeling with extremely low robustness specialized for only that audio sample is performed (over-learning). Therefore, in order to ensure the robustness of the acoustic model, model parameters are often shared between different acoustic units. The sharing of model parameters increases the amount of speech samples per parameter, resulting in relatively high reliability of parameter estimates, especially for acoustic units that do not appear very often in speech samples. Robustness as a whole is improved. However, sharing the model parameters means, for the most part, that acoustic units sharing parameters are not identified at least in part. May sacrifice precision.

[0003] As described above, precision and robustness have a relationship in that if one is pursued, the other will be impaired, and it is important to find the equilibrium point between the two as appropriately as possible in accordance with the amount of audio samples. It is. By the way, this equilibrium point will also differ depending on the type of language model combined with the acoustic model. For example, a case where only a phoneme connection rule is used as a language model and a case where a word N-
It seems that the role of the acoustic model should be different from the case where the gram is used.

In the former case, the demand for the phoneme identification capability of the acoustic model, that is, the precision, will increase. This is because if the alternative identification is not successful in the acoustic model, the possibility of recovering the error only by the phoneme connection rule is limited. On the other hand, in the latter case, even if alternative identification is not necessarily successful in the acoustic model, the error can be recovered compared to the former, due to the phoneme arrangement that can be accepted as a vocabulary, the restriction on the word connection probability, etc. Sex is big. In other words, when relatively strong language constraints such as the latter are used, the acoustic model has higher precision, that is, “the highest likelihood for the corresponding phoneme,” as compared to the case of loose language constraints. Rather than "giving degree and giving low likelihood to other phonemes", i.e. "more distribution gives a reasonably high likelihood for the corresponding phoneme That is, it is sought, and therefore, at the expense of "giving the highest likelihood to the corresponding phoneme and giving low likelihood to other phonemes",
Overall, it is thought to give good results.

[0005] Meanwhile, as a prior study of sharing a Gaussian distribution among a plurality of Gaussian mixture distributions in an HMM, for example, in prior art document 1, "XDHuang et al.," Unified Tec "
hniques for Vector Quantization and Hidden Markov
Modeling Using Semi-continuous models ", Proceddings
of ICASSP'89, pp. 639-642, 1989, etc. (hereinafter referred to as a first conventional example) is representative. In addition, as a method of determining the distribution sharing relationship based on a voice sample, a sequential state-division fusion method (for example, Prior Art Document 2 “Junichi Takami,“ Automatic creation of a highly efficient hidden Markov network by the state-division fusion method ”) , IEICE Transactions (D-II), J78-D-II, No.
5, pp. 717-726, May 1995 ". )
(Hereinafter referred to as a second conventional example).

[0006]

However, the sharing relationship according to the first conventional technique is "determined based on a parametric distance between Gaussian mixture distributions, and does not reflect the characteristics of actual speech samples." There is a disadvantage that. Further, in the second conventional example, (a) after temporarily determining a sharing relationship based on a parametric distance between Gaussian mixture distributions and then performing only final adoption or rejection using a voice sample, the voice is not necessarily used. There is no guarantee that a shared structure based on sample characteristics will be formed. (B) Since the sharing of the Gaussian distribution is performed only in units of the entire component of the Gaussian mixture distribution, the expression of the distribution lacks. (C) Basically, each time a fusion decision for a state
Since the parameters of the entire M are re-estimated, it takes a long time to finally determine the sharing relationship. There is a disadvantage that.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned problems, to more accurately reflect the characteristics of voice data, and to improve the expressive power of an unspecified speaker acoustic model as compared with a conventional example. Another object of the present invention is to provide an acoustic model generation device and a speech recognition device for generating an acoustic model capable of performing speech recognition at a higher speech recognition rate.

[0008]

According to a first aspect of the present invention, there is provided an acoustic model generating apparatus which performs a predetermined learning algorithm based on a characteristic parameter of predetermined voice data.
First generating means for generating an initial Hidden Markov Model; and a Hidden Markov Model based on a tendency of a frame error, which is an identification error of a frame unit at a predetermined time, caused by the initial Hidden Markov Model for the audio data. By adding the components of the Gaussian mixture of
A second generation unit configured to reconstruct an initial hidden Markov model generated by the first generation unit and generate a reconstructed hidden Markov model; And a third generation unit configured to re-learn the hidden Markov model generated by the second generation unit by a learning algorithm, thereby generating an acoustic model that is a re-learned hidden Markov model. And

According to a second aspect of the present invention, in the acoustic model generating apparatus according to the first aspect of the present invention, the second generating means includes a Viterbi between the initial hidden Markov model and the audio data. By executing the alignment processing, the highest likelihood is given to (a) the Viterbi sequence of the Gaussian mixture distribution included in the initial hidden Markov model and (b) each frame of the audio data, and First processing means for obtaining a maximum likelihood sequence of a Gaussian distribution included as a component of each Gaussian mixture distribution in the hidden Markov model of, and a Viterbi sequence of a Gaussian mixture distribution obtained by the first processing means; In the maximum likelihood sequence of the Gaussian distribution, based on the respective appearance frequencies of the Gaussian mixture distribution at the same time and the combination of the Gaussian distributions, For all combinations of the Gaussian mixture distribution and the Gaussian distribution included in the hidden Markov model of the period, when a frame error occurs in each Gaussian mixture distribution and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution of the combination Second processing means for calculating an error probability, and adding the Gaussian distribution as a new component of the Gaussian mixture distribution when each calculated frame error probability exceeds a predetermined threshold value,
Each Gaussian distribution added as a new component of each Gaussian mixture distribution by the second processing means includes a Gaussian mixture distribution whose Gaussian distribution belongs as a component in the initial hidden Markov model and the second Gaussian distribution. And a Gaussian mixture distribution newly assigned as a component by the processing means.

Further, a speech recognition apparatus according to the present invention uses the acoustic model generated by the acoustic model generation apparatus according to claim 1 or 2 to perform speech recognition based on a speech signal of an input uttered speech sentence. It is characterized by having voice recognition means.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a continuous speech recognition apparatus according to an embodiment of the present invention. In the present embodiment,
In a speech recognizer using a word-unit N-gram, its robustness is based on the assumption that a trained acoustic model (HMM) is in a state of excessive precision in speech recognition direction with loose language constraints. , The Gaussian distribution is shared among a plurality of Gaussian mixture distributions in the learned HMM. From a modeling standpoint, it is desirable that this shared structure be determined based on audio samples. In this embodiment,
In order to solve the problems of the prior art, (a) determining a Gaussian distribution sharing relationship based only on audio data,
(B) The Gaussian distribution is shared for each Gaussian distribution, and (c) the parameter reestimation for determining the shared structure is 1
A method of sharing a Gaussian distribution among a plurality of Gaussian mixture distributions, which can determine a sharing relationship in a short time, is used because the sharing is performed only once.

As shown in FIG. 1, the speech recognition apparatus according to the present embodiment comprises: (a) an initial HMM using a predetermined learning algorithm based on characteristic parameters of predetermined voice data stored in a voice data memory 30; Initial HM that generates
(B) adding a component of the Gaussian mixture distribution of the HMM based on the tendency of a frame error, which is an identification error in a frame unit at a predetermined time, caused by an initial HMM to the audio data; The initial H
An HMM reconstructing unit 22 that reconstructs the initial HMM generated by the MM generating unit 21 and generates a reconstructed HMM; and (c) a predetermined learning algorithm based on the feature parameter of the audio data. , A re-learning unit 23 that re-learns the HMM generated by the HMM reconstructing unit 22 to generate an acoustic model that is the re-learned HMM.

Here, the HMM reconstructing unit 22 calculates (b1)
By performing a Viterbi alignment process between the initial HMM and the audio data, the initial HMM
After obtaining a set of a plurality of Gaussian mixture distributions and a set of a plurality of Gaussian distributions respectively included in (b2), for each combination of the obtained plurality of Gaussian mixture distributions and the plurality of Gaussian distributions, Calculate the frame error probability when a frame error occurs and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution of the combination,
When each of the calculated frame error probabilities exceeds a predetermined threshold value, the Gaussian distribution is added as a new component of the Gaussian mixture distribution.

The speech recognition apparatus shown in FIG. 1 uses the phoneme HMM generated by the relearning unit 23 to perform speech recognition based on the speech signal of the input uttered speech sentence. Here, the speech recognition device of the present embodiment is a known one-pass
A word collating unit 4 for detecting the word hypothesis of the uttered speech sentence based on the feature parameter of the speech signal of the input uttered speech sentence and calculating and outputting the acoustic likelihood using the Viterbi decoding method.
In the continuous speech recognition device provided with the above, for the word hypothesis output from the word matching unit 4 via the buffer memory 5, the peak of the acoustic likelihood of the central part of each phoneme of the word in the time direction is calculated. A likelihood correction unit 7 that delays the movement to a time delayed from the center to correct the acoustic likelihood of the word hypothesis, and a total likelihood including the acoustic likelihood output from the likelihood correction unit 7. Based on a word hypothesis having a degree, for each head phoneme environment of the word, one word hypothesis having the highest likelihood among the total likelihoods calculated from the utterance start time to the end time of the word is represented. A word hypothesis narrowing unit 6 that narrows down word hypotheses so as to cause the word hypothesis to be provided.

Hereinafter, the HMM generation processing will be described in detail. The voice data memory 30 previously stores feature parameters of voice data of an unspecified speaker, and the feature parameters are obtained by LPC analysis of voice samples obtained by A / D conversion of a voice waveform signal on a frame basis. Log power, 16th-order LPC cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. The initial HMM generation unit 21 uses a Baum-Welch learning algorithm based on the characteristic parameters of the predetermined voice data stored in the voice data memory 30.
An initial HMM is generated and stored in the initial HMM memory 31.

Next, the HMM reconstructing unit 22 reconstructs the Gaussian mixture distribution (component addition and sharing) of the initial HMM based on the speech data in the speech data memory 30 by the method of the present embodiment, and reconstructs the HMM. The configured HMM is stored in the reconfigured HMM memory 32. Further, the re-learning unit 23 re-estimates the parameters of the reconstructed HMM, and stores the parameters in the phoneme HMM memory 11 as the final phoneme HMM. In the reconstruction method and parameter re-estimation of the present embodiment, basically, the speech data used for creating the initial model is used as it is. Therefore, there is an advantage that it is not necessary to prepare a new audio sample for this processing.

Further, the concept of adding a component in the HMM reconfiguration processing will be described. The essence of the method of the present embodiment is to add components of the Gaussian mixture distribution in consideration of the tendency of frame-by-frame identification errors (hereinafter, frame errors) caused by the initial HMM for voice data. This frame error is the initial HM
Viterbi alignment (Vite
rbi alignment) processing when the following conditions are satisfied.

(Equation 1)

Here, the function max (·) is a function indicating the value at which the function value becomes maximum when the variable γ (Gaussian mixture distribution) changes when γ∈Γ. Further, o _t: feature vector at time t, g _t: Gaussian mixture distribution assigned to the time t by the Viterbi alignment process, gamma: set of initial HMM entire Gaussian mixture (i.e., a plurality in each state of the initial HMM It is a set of Gaussian mixture distributions.), P (o│g): Likelihood that the feature vector o is output from the distribution g.

Hereinafter, a frame error is represented by an event E.
Is written as E ^c . The Gaussian mixture distribution that frequently causes frame errors for a specific acoustic phenomenon tends to cause a drop in acoustic likelihood when performing matching with the acoustic phenomenon on the correct path in actual speech recognition. Consider the following Gaussian distribution x _t at time t.

(Equation 2) Here, the function argmax is a function indicating the variable の when the function value becomes maximum when the variable ξ (Gaussian distribution) changes when ξ∈Ξ. Ξ: Initial HMM
This is a set of the entire Gaussian distribution (that is, a set of Gaussian distributions that are the basis of the Gaussian mixture distribution in each state in the initial HMM).

X _t is originally a component of any Gaussian mixture distribution in the entire HMM, but is treated here as a single Gaussian distribution that gives the maximum acoustic likelihood to the feature vector o _i . Furthermore, the original time series Γg of Γ
For the time series {x _t} of the original _t} and .XI, by analyzing the frequency of occurrence of each element, the conditional frame error probability, P
(E, ξ | γ) (γ∈Γ, ξ∈Ξ) is obtained. The conditional frame error probability P (E, ξ | γ) is a frame error probability when a frame error occurs in the Gaussian mixture distribution ξ and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution γ. Frame error probability P (E, ξ | γ)
Has a somewhat large value, the Gaussian mixture distribution γ
Is likely to cause a drop in acoustic likelihood when performing matching with acoustic phenomena near the Gaussian distribution ξ. Therefore, it is considered that the adverse effect can be suppressed by newly adding the Gaussian distribution ξ as a component of the Gaussian mixture distribution γ.

Next, the algorithm of the HMM reconstruction processing will be described. By executing the following processes of (Step SS1) to (Step SS5), addition and sharing of components of the Gaussian mixture distribution are realized. (Step SS1) perform Viterbi alignment process between the initial HMM and the voice sample, the time series of the original Γ is a set of Gaussian mixture distributions, i.e. Viterbi sequence {g _t},
And the original time series of Ξ which is a set of Gaussian distributions, that is, the maximum likelihood sequence {x _t } of the Gaussian distribution, respectively (see FIG. 7). (Step SS2) From the time series obtained in step SS1,
The conditional frame error probability P (E, ξ | γ) is obtained for all combinations of the Gaussian mixture γ and the Gaussian ξ (see FIG. 8). (Step SS3) Step SS4 is executed for all Gaussian mixture distributions γ. (Step SS4) Step SS5 is executed for all Gaussian distributions ξ. (Step SS5) Conditional frame error probability P (E,
If ξ | γ) exceeds a predetermined threshold value, the Gaussian mixture distribution ξ is added as a new component of the Gaussian distribution γ (see FIG. 9). The added component is shared between the Gaussian distribution γ and the Gaussian mixture distribution to which the Gaussian mixture distribution ξ originally belonged. By the threshold processing in step SS5, it is possible to suppress the addition of components for frame errors caused by accidental noise or the like included in audio data. That is, the HMM reconstruction processing is performed by (I) the initial HM
By executing the Viterbi alignment process between M and the audio data, the highest Viterbi sequence of (a) the Gaussian mixture distribution included in the initial HMM and (b) each frame of the audio data are obtained. A first process of giving a likelihood and obtaining a maximum likelihood sequence of a Gaussian distribution included as a component of each Gaussian mixture distribution in the initial HMM;
(II) In the Viterbi sequence of the Gaussian mixture distribution and the maximum likelihood sequence of the Gaussian distribution obtained by the first processing,
Based on the appearance frequency of each combination of the Gaussian mixture distribution and the Gaussian distribution at the same time, the initial HM
For all combinations of Gaussian mixture distribution and Gaussian distribution included in M, calculate the frame error probability when a frame error occurs in each Gaussian mixture distribution and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution of the combination. And second processing means for adding the Gaussian distribution as a new component of the Gaussian mixture distribution when each of the calculated frame error probabilities exceeds a predetermined threshold value, and (III) the second processing means The Gaussian distribution added as a new component of each Gaussian mixture distribution by the above process is the Gaussian mixture distribution that the Gaussian distribution belonged to as a component in the initial HMM, and the Gaussian mixture distribution newly added as a component by the second process. It is a component that is shared by both the Gaussian mixture distribution that came to belong.

FIGS. 3 to 6 show the HMM reconstructing unit 2 shown in FIG.
6 is a flowchart illustrating details of an HMM reconfiguration process performed by the HMM 2. First, in step S1 of FIG. 3, the parameter of the audio data number n is initialized to 0.
In step S2, Viterbi alignment processing is performed between the audio data #n and the initial HMM, and a Viterbi sequence {g ⁿ t} of a Gaussian mixture distribution and a maximum likelihood sequence {x ⁿ t} of a Gaussian distribution.
Ask for. Then, in step S3, it is determined whether or not the processing of step S2 has been performed for all data. If NO, the parameter n is incremented by 1 in step S4, and then the processing of step S2 is executed. On the other hand, if YES in step S3, the process proceeds to step S11 in FIG.

In step S11 of FIG. 4, all the count values C (•) are initialized to 0. In step S12, the parameter of the audio data number n is initialized to 0.
After the parameter of the frame number t is initialized to 0 in step S14, it is determined in step S14 whether or not a frame error has occurred. If a frame error has occurred, the count value C (E) for counting the frame error in step S15. , G ⁿ t | x ⁿ t) is incremented by one, and the process proceeds to step S17. On the other hand, if no frame error has occurred in step S14, the count value C (E ^c | g ⁿ t) for counting that no frame error has occurred is incremented by one in step S16, and the flow proceeds to step S17. In step S17, step S1 is performed for all frames of the audio data #n.
It is determined whether or not the process of step 4 has been performed. If the determination is NO, the parameter t is incremented by 1 in step S18, and the process returns to step S14. On the other hand, in step S17, YE
In the case of S, it is determined in step S19 whether or not the processing of step S14 has been performed for all the audio data.
In the case of, the parameter of the data number n is incremented by 1 in step S20, and the process returns to step S13. On the other hand, if YES in step S19, the process proceeds to step S21 in FIG.

In step S21 of FIG. 5, the parameter of the number i of the Gaussian mixture distribution is initialized to 0, and step S2
After the parameter of the Gaussian distribution number j is initialized to 0 in step 2, the conditional frame error probability P (E, γ _i | ξ _j ) is calculated in step S23 using the following equation.

(Equation 3) Then, in step S24, it is determined whether or not the processing in step S23 has been performed for all Gaussian distributions. If NO, the parameter j is incremented by 1 in step S25, and the process returns to step S23. On the other hand, step S24
If YES in step S26, it is determined in step S26 whether or not the processing in step S23 has been performed for all Gaussian mixture distributions. If NO, the parameter i is set to 1 in step S27.
And the process returns to step S22. on the other hand,
If YES in step S26, the process proceeds to step S3 in FIG.
Proceed to 1.

In step S31 of FIG. 6, the parameter of the number i of the Gaussian mixture distribution is initialized to 0, and step S3
After the parameter of the number j of the Gaussian distribution is initialized to 0 in step 2, in step S33, the conditional frame error probability P
When (E, γ _i | ξ _j ) exceeds the threshold value ρ (0.01 in the preferred embodiment), the Gaussian mixture distribution ξ is added as a new component of the Gaussian distribution γ. Proceeds, while N is determined in step S33.
If it is O, the process proceeds directly to step S35. In step S35, it is determined whether or not the process in step S33 has been performed for all Gaussian distributions. If NO, the parameter j is incremented by 1 in step S36, and the process returns to step S33. On the other hand, if YES is determined in the step S35, it is determined in a step S37 whether or not the process of the step S33 is performed for all Gaussian mixture distributions.
In step S38, the parameter i is incremented by 1 in step S38, and the process returns to step S32. On the other hand, step S
If YES in step S37, the reconfigured HMM obtained in step S39 is stored in the memory 32, and the HMM reconfiguration processing ends.

Further, after reconstructing the HMM, the re-learning unit 23 re-estimates the following parameters based on criteria such as maximum likelihood and maximum likelihood ratio using, for example, a Balm-Welch learning algorithm. And the phoneme HMM after re-learning
Is stored in the phoneme HMM memory 11. (A) Average of each Gaussian distribution, (b) Variance of each Gaussian distribution, (c) Mixing weight of each Gaussian mixture distribution, (d) State transition probability

As for the mean, variance, and state transition probability of the Gaussian distribution, the values of the initial HMM are used as they are as initial values. For the mixture weight of the Gaussian mixture distribution,
Considering the frame error probability and the threshold value of the component addition execution, the initial value is determined as follows.

First, the initial value of the mixture weight when the Gaussian distribution ξ is added as a new component to the Gaussian mixture distribution γ is expressed by the following equation using the value of the conditional frame error probability as it is.

## EQU4 ## wh ^γ _ξ = P (E, ξ | γ)

With the addition of the component,
A new initial value is also required for the mixture weight of the components originally included in the initial HMM. These are given by the following equations.

Wh ^γ _ξ = P (E ^c | γ) + P (E, Ξ ^γ _ρ
| ^Γ)} · w γ _ξ However, wh ^gamma _xi], in the initial HMM, a mixture weights in the Gaussian mixture distributions gamma Gaussian xi]. Here, Ξ ^γ _ρ is determined when the threshold for component addition execution is ρ
Is a set of Gaussian distributions not subject to component addition to the Gaussian mixture distribution γ, given by

６ ^γ _ρ = ｛ξ | P (E, ξ | γ) <ρ; ξ∈Ξ｝

The likelihood correction of the likelihood correction unit 7 used in this embodiment can be called a beam search of a delayed decision. The beam search for this delay determination is
The local variation of likelihood is dealt with by delaying the evaluation of the likelihood of a route that has already been searched, without relying on likelihood look-ahead and the likelihood mapping by a nonlinear function as in the fourth conventional example. I do. In the following calculation, the likelihood indicates the log likelihood. In the present embodiment, each code is defined only in the likelihood correction unit 7 as follows. (A) t: time; (b) S: path of beam search; (c) q _A (S, t): path S, acoustic likelihood at time t; (d) Q _A (S, t): path S, cumulative acoustic likelihood from beginning of a sentence at time _{t; (e) Q L (} S, t): path S, the cumulative language likelihood from beginning of a sentence at time t.

Here, the acoustic likelihood is a likelihood calculated by the word matching unit 4 with reference to the phoneme HMM in the phoneme HMM memory 11, and the language likelihood is calculated by the statistical language model in the word matching unit 4. The likelihood is calculated with reference to the language model in the memory 13. When defined as described above, generally, the cumulative acoustic likelihood is obtained by adding the acoustic likelihood for each frame by the following equation.

Q _A (S, t) = Q _A (S, t−1) + q _A (S, t)

The cumulative total likelihood Q _all (S, t) from the head of the sentence used for beam search is the acoustic likelihood Q _A (S,
t) and the language likelihood Q _L (S, t) are calculated by the following equation.

[Equation 8] _{Q all (S, t) =} Q A (S, t) + α · Q L (S, t)

Here, the constant α is a weight coefficient for the sound likelihood of the language likelihood, and in a preferred embodiment, α
= 4.5. In the beam search for delay determination in the present embodiment, as shown in the following equation,
Using the _A (S, t) likelihood Q _A '(S, t) obtained by subtracting the Q _A (S, t) delayed from the acoustic likelihood Q _Ad (S, t) instead of. That is, at time t-1, as shown in the following equation, the delayed acoustic likelihood Q _Ad (S, t-1) is obtained from Q _A (S, t-1) instead of Q _A (S, t-1). Likelihood Q minus
_A ′ (S, t−1) is used.

[0035]

Q _A ′ (S, t) = Q _A (S, t) −Q _Ad (S, t) Here, the likelihood Q _Ad (S, t) of the second term on the right side of the above equation 9
Is calculated by the following equation.

D = Q _Ad (S, t-1) + q _A (S, t)

## _EQU11 ## Q _Ad (S, t) = F (D) .D

By rewriting equation 9 and rewriting with reference to equation 7, the following equation is obtained.

[Number 12] _{Q A '(S, t)} = Q A' (S, t-1) + q
_A ′ (S, t) Here, the likelihood q _A ′ (S, t) is determined by the following equation.

Equation 13] _{q A '(S, t)} = f (x) = f (q A (S, t) + Q A (S, t-1) -Q A' (S, t-1)

Here, ｛Q _A (S, t) in the above equation (13)
-1) -Q _A '(S, t-1)} is Q _Ad (S, t-1), which is disclosed in Japanese Patent Application Laid-Open No. 9-811 by the present applicant.
This data is an underestimated value one time earlier than that of the embodiment of JP-A-85-85, and this data is sequentially stored in an underestimated likelihood memory 14 connected to the likelihood correction unit 7, and is stored at the next time t. It is used to calculate the total likelihood by correcting the acoustic likelihood. Therefore, in the present embodiment, at time (t−1), the likelihood correction unit 7 calculates the ｛Q _A (S, t-1) -Q _A '(S, t-1)} is calculated and stored in the underestimated likelihood memory 14, and then at time t
In the above, the acoustic likelihood Q _A ′ (S, t) corrected so as to be underestimated is calculated using the above equations 12 and 13, and then the following equation obtained by rewriting the above equation 8 is used. hand,
Calculate the total likelihood Q ′ _all (S, t), which is the cumulative likelihood,
The word hypothesis having the calculated overall likelihood Q ′ _all (S, t) is output to the word hypothesis narrowing unit 6 via the buffer memory 5.

Q ′ _all (S, t) = Q _A ′ (S, t) + α ·
Q _L (S, t)

In the above equation (13), the function f (x)
Is a first function for calculating a delay ratio with respect to the likelihood x. For example, the function x is a function that generally changes so as to decrease the slope of the function f (x) as x increases. Further, the function F (D) in the above equation 11 is a second function related to the first function and for calculating a delay ratio with respect to the likelihood D.

Next, the configuration and operation of the continuous speech recognition apparatus of FIG. 1 will be described. In FIG. 1, a phoneme HMM memory 11 is connected to the word matching unit 4 and stores a phoneme HMM in advance, and the phoneme HMM is represented including each state,
Each state has the following information. (A) State number (b) Acceptable context class (c) List of preceding state and succeeding state (d) Parameter of output probability density distribution (e) Self transition probability and transition probability to succeeding state The phoneme HMM used in the example is created by converting a predetermined speaker mixed HMM because it is necessary to specify which speaker each distribution originates from. Here, the output probability density function is a Gaussian mixture distribution having a 34-dimensional diagonal covariance matrix.

The word dictionary memory 12 is connected to the word collating unit 4 and stores a word dictionary in advance. The word dictionary stores a reading of each phoneme HMM in the phoneme HMM memory 11 represented by a symbol for each word. The symbol string shown is stored. Further, the statistical language model memory 13 is connected to the word matching unit 4 and stores a predetermined statistical language model in advance. Here, the statistical language model is described in, for example, the related art document 6
"Hirokazu Masataki et al.," Variable Length Statistical Language Model for Continuous Speech Recognition ", IEICE Technical Report, SP95
-73, November 1995 ".
A language model called variable length N-gram whose length in the time direction is variable can be used. The statistical language model includes a variable length N-gram of a part of speech class and a word.
And expressed as a bigram between the following three types of classes. (A) part-of-speech class, (b) class of word separated from part-of-speech class, and (c) class formed by connecting connected words.

In the continuous speech recognition apparatus of FIG. 1, the feature extracting unit 2, the word matching unit 4, the likelihood correcting unit 7, the word hypothesis narrowing unit 6, the initial HMM generating unit 21, and the HMM reconstructing unit The re-learning unit 23 and the re-learning unit 23 are configured by, for example, a digital computer including a CPU. Also, the buffer memory 3,
5, phoneme HMM memory 11, and word dictionary memory 12
, A statistical language model memory 13, an underestimated likelihood memory 14, a voice data memory 30, an initial HMM memory 31, and a reconstructed HMM memory 32,
It consists of a hard disk memory.

In FIG. 1, a uttered voice of a speaker is input to a microphone 1 and converted into a voice signal, and then input to a feature extracting unit 2. After performing A / D conversion on the input audio signal, the feature extraction unit 2 performs, for example, LPC analysis, and performs 34-dimensional feature parameters including logarithmic power, 16th-order cepstrum coefficient, Δlogarithmic power, and 16th-order Δcepstrum coefficient. Is extracted. The time series of the extracted feature parameters is input to the word matching unit 4 via the buffer memory 3.

The word collating unit 4 uses the one-pass Viterbi decoding method to store the phoneme HMM memory 1 based on the characteristic parameter data input via the buffer memory 3.
1, a word hypothesis is detected using the word dictionary in the word dictionary memory 12, and the statistical language model in the statistical language model memory 13, and the acoustic likelihood based on the phoneme HMM, And calculates the linguistic likelihood based on the statistical language model, and outputs the linguistic likelihood together with the word hypothesis to the likelihood correcting unit 7. Here, the word matching unit 4 determines, for each state of each HMM at each time,
The likelihood within a word and the acoustic likelihood from the start of utterance are calculated. The likelihood including the acoustic likelihood and the linguistic likelihood is individually provided for each word identification number, word start time, and preceding word difference. Also,
In order to reduce the amount of calculation processing, the grid hypothesis of a low total likelihood among the total likelihoods calculated based on the phoneme HMM, the word dictionary, and the statistical language model is reduced. The word matching unit 4 outputs the resulting word hypothesis and information on the overall likelihood to the likelihood correction unit 7 together with time information (specifically, for example, a frame number) from the utterance start time.

In response, at time (t-1), likelihood correction section 7 calculates the ｛Q _A (S , T
-1) -Q _A ′ (S, t−1)} is stored in the underestimated likelihood memory 14, and at time t, the underestimated is calculated using the above equations 6 and 7. The sound likelihood Q _A ′ (S, t) corrected so as to calculate the total likelihood Q ′ _all (S, t) by using the above equation (8),
The word hypothesis having the calculated overall likelihood Q ′ _all (S, t) is output to the word hypothesis narrowing unit 6 via the buffer memory 5.

The word hypothesis narrowing section 6 is based on a word hypothesis having an overall likelihood output from the likelihood correction section 7 via the buffer memory 5, and is configured to select words having the same end time but different start times from each other. With respect to the hypothesis, the word is represented by one word hypothesis having the highest likelihood among the calculated total likelihoods from the utterance start time to the end time of the word for each head phoneme environment of the word. After narrowing down hypotheses, of the word strings of all the narrowed word hypotheses,
A word string of a hypothesis having the maximum overall likelihood is output as a recognition result. In the present embodiment, preferably, the first phoneme environment of the word to be processed is three phonemes including the last phoneme of the word hypothesis preceding the word and the first two phonemes of the word hypothesis of the word. I mean a line.

[0046] For example, as shown in FIG. 2, the (i-1) th word W _i-1 of the following phoneme string a _1, a _2, ..., come i th word W _i consisting a _n Sometimes, six hypotheses Wa, Wb, Wc, Wd, We, and Wf are assumed as the word hypotheses of the word Wi _-1.
Exists. Here, the former three word hypotheses Wa, W
It is assumed that the final phonemes of b and Wc are / x /, and the final phonemes of the latter three word hypotheses Wd, We and Wf are / y /. Of the hypotheses in which the end time t _e is equal to the head phoneme environment (in FIG. 2, the top three word hypotheses whose head phoneme environment is “x / a ₁ / a ₂ ”), the hypothesis with the highest overall likelihood (for example, FIG. 2 except for the top hypothesis). Note that the fourth hypothesis from the top has a different phoneme environment, that is,
Since the last phoneme of the preceding word hypothesis is y instead of x, the fourth hypothesis from the top is not deleted. That is, only one hypothesis is left for each final phoneme of the preceding word hypothesis. FIG.
In the example, leave one hypothesis for the final phoneme / x /
Leave one hypothesis for the final phoneme / y /.

In the above embodiment, the head phoneme environment of the word is defined as a sequence of three phonemes including the last phoneme of the word hypothesis preceding the word and the first two phonemes of the word hypothesis of the word. Although defined, the present invention is not limited to this. The phoneme sequence of the preceding word hypothesis including the final phoneme of the preceding word hypothesis, and at least one phoneme of the preceding word hypothesis that is continuous with the final phoneme, And a phoneme sequence that includes a phoneme sequence that includes the first phoneme of the word hypothesis.

[0048]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present inventor conducted the following experiment in order to confirm the effectiveness of the speech recognition apparatus shown in FIG. The Gaussian mixture distribution reconstruction of the learned HMM was performed by the above method.
As the initial HMM, a naturally spoken voice of 175 male speakers in a corpus (text data) of a travel setting owned by the patent applicant is used as a learning voice sample, and a known ML-
The unspecified speaker HMNet for male speakers created by the SSS algorithm was used. This HMNet expresses the phoneme HMM depending on the environment before and after by a state sharing network. Tables 1 and 2 show the conditions for HMNet used in the initial HMM.

[0049]

[Table 1] Acoustic analysis conditions サ ン プ リ ン グ Sampling frequency: 12 kHz Quantization: 16-bit linear Pre-emphasis: 1 −0.97z ⁻¹ window: 20 ms Hamming frame shift: 10 ms Feature vector: log power + 16th order LPC cepstrum coefficient + Δlog power + 16th order Δcepstrum coefficient ────────────────── ───────────

[0050]

Table 2 Conditions for the structure of HMNet ───────────────────────────── (a) Male speaker independent HMet state in 401 state 400 states for a plurality of divided phoneme HMMs (3 context-dependent HMMs) 1 state for a silence HMM (b) Acoustic unit: 25 Japanese phonemes + silence (c) Mixed size: 10 mixtures / state (d) Covariance Type: orthogonal ─────────────────────────────

For this HMNet, the Gaussian mixture distribution was reconstructed at the threshold value r = 0.01 of the component addition execution using the natural speech voice of the 175 male speakers described above. As a result, a total of 2603 components are added to the total number of components 4000 in the initial HMM, and the mixture number of each Gaussian mixture distribution, which was uniformly 10 in the initial HMM, is 10 to 10.
It was distributed in a range of 36 (see FIG. 10).

Next, when examining the relationship between the addition source and the addition destination of the Gaussian mixture distribution in component addition, it is found that distributions expressing the same central phoneme are performed for more than half of the component addition. Was.
HMNet used in this experiment is an allophone HMM for each central phoneme.
Are sequentially divided in the environmental direction or the time direction. Therefore, the addition of components between distributions expressing the same central phoneme mainly serves to make it possible to express again an acoustic phenomenon that cannot be expressed in any state due to sequential state division in the HMNet creation process. Can be considered. It was also confirmed that such component addition was performed in both the environment direction and the time direction.

In the parameter re-estimation process, the parameters of the HMM after the Gaussian mixture distribution reconstruction were re-estimated. When estimating parameters,
By properly selecting the criterion, it can be expected to maximize the effect of the reconstruction, but in this experiment,
In order to evaluate the effect of the change in the structure of the Gaussian mixture distribution due to the reconstruction, the maximum likelihood criterion used when the initial HMM was created was adopted. Using the spontaneously uttered speech by 175 male speakers as in the creation of the initial HMM and the distribution reconstruction, Baum-Welc
The following parameters were estimated by the learning algorithm of h). (A) mean of Gaussian distribution, (b) variance of Gaussian distribution,
(C) Gaussian mixture distribution mixture weight, and (d) state transition probability.

Next, a continuous speech recognition experiment will be described. A continuous speech recognition experiment was performed using a reconstructed HMM obtained by Gaussian mixture distribution reconstruction and subsequent parameter re-estimation.
It is called a comparative example. ) And their recognition rates. The experimental conditions are shown below. (A) Continuous speech recognizer: A continuous speech recognizer characterized by multipath search and word graph output (see FIG. 1). (B) Language model: variable-length word class N-gram, number of separated classes: 500. (C) Word dictionary: 6922 vocabulary words (d) Test data: travel conversation voices for seven male open speakers, corpus (text data) of travel settings owned by the patent applicant, 81 utterances, 937 words in total. (E) Evaluation Criteria Word accuracy and word% correct for the first recognition candidate in the word graph, defined by the following equation.

## EQU15 ## Word accuracy = (NIDS) / N

Where: N: number of correct words, I: number of insertion errors, D: number of missing errors, S: number of replacement errors.

The beam width and the linguistic likelihood weight of the continuous speech recognition apparatus were set by a preliminary experiment so that the word accuracy was maximized in the speech recognition using the initial HMM. FIG. 11 shows a recognition result when only the weight coefficient α for the acoustic likelihood of the language likelihood defined above is changed from the optimal setting for the initial HMM. It can be seen that the reconstructed HMM (Example) after re-learning exceeds the initial HMM (Comparative Example) for both word accuracy and word% correct.

In the present embodiment, the expression power of the learned Gaussian mixture distribution type unspecified speaker HMM for the purpose of suppressing the local drop of the acoustic likelihood is determined by reconstructing the Gaussian mixture distribution using speech samples. Invented a method to achieve this. This method of adding and sharing a component based on the error tendency obtained by matching a learned HMM with a speech sample can effectively suppress a local drop in acoustic likelihood, and as a result, speech recognition It was confirmed that the rate improved.

Further, the effect of the reconstructed HMM after relearning of the present embodiment according to the present invention will be considered below. (A) Expressive power of distribution The expressive power of a distribution is determined by the number of mixtures of each Gaussian mixture distribution. In the successive state division fusion method, the number of mixtures becomes equal for all Gaussian mixture distributions, and a structure that can correspond to the fineness of an object to be expressed for each Gaussian mixture distribution is not generated. In the present invention, various mixtures of Gaussian mixture distributions ranging from 5 mixtures (the number of mixtures in the initial model) to 36 mixtures are generated in the embodiment. In addition, since the total number of Gaussian distributions is the same as before the application by the shared structure of distribution, the expressiveness of the distribution can be enhanced while maintaining the robustness as an acoustic model. (B) Calculation Time for Determining the Shared Structure Most of the time required for determining the final shared structure occupies the time required for calculating the likelihood of the acoustic model for the learning data. In the sequential state division fusion method of the second conventional example, in order to re-estimate the parameters of the entire HMM every time fusion of one state is determined, when the total number of states of the model is N,

[Mathematical formula-see original document] The likelihood calculation of N + N = 2 * N (times) is required. Here, the first term on the left side of Expression 17 is a likelihood calculation for each state division. In the embodiment of the present invention, since the shared structure for all states is determined collectively, the likelihood calculation required for determining the shared structure itself is 2
Times. Therefore,

## EQU18 ## The likelihood calculation is N + 2 (times). Here, the first term on the left side of Expression 18 is a likelihood calculation for each state division. Usually N is 400 to 1000
, The calculation time is expected to be reduced by almost half.

As described above, according to the present embodiment, for the HMM that has been re-learned after the initial HMM has been reconfigured by adding components as described above, the initial HM
Compared with M, the expression power of the distribution can be enhanced while maintaining the robustness as an acoustic model. Therefore, the HM
By performing voice recognition using M, voice recognition can be performed at a higher voice recognition rate than in the related art. Further, the calculation time for determining the shared structure can be reduced to almost half as compared with the second conventional example, and the HMM can be constructed at a higher speed.

[0059]

As described above in detail, according to the acoustic model generating apparatus according to the first aspect of the present invention, an initial hidden Markov model is obtained by a predetermined learning algorithm based on a characteristic parameter of predetermined speech data. And a Gaussian mixture distribution of the Hidden Markov Model based on a tendency of a frame error, which is an identification error of a frame unit at a predetermined time, caused by an initial Hidden Markov Model for the audio data. Second generating means for reconstructing the initial hidden Markov model generated by the first generating means by adding components to generate a reconstructed hidden Markov model; and characteristics of the audio data The hidden Markov model generated by the second generation unit based on the parameters and by a predetermined learning algorithm By relearning, third generating an acoustic model is re-learned HMM
Generating means. Therefore, as for the HMM obtained by reconstructing the initial HMM by adding components as described above and then re-learning, the expression power of the distribution can be enhanced while maintaining the robustness as an acoustic model, as compared with the initial HMM. . Therefore, by performing voice recognition using the HMM, voice recognition can be performed at a higher voice recognition rate than in the related art. Further, the calculation time for determining the shared structure can be reduced to almost half as compared with the second conventional example, and the HMM can be constructed at a higher speed.

According to a second aspect of the present invention, in the acoustic model generating apparatus as set forth in the first aspect, the second generating means is configured to perform a process between the initial hidden Markov model and the audio data. By executing the Viterbi alignment processing, (a) a Viterbi sequence having a Gaussian mixture distribution included in the initial hidden Markov model,
(B) first processing means for giving the highest likelihood to each frame of the audio data and obtaining a Gaussian maximum likelihood sequence included as a component of each Gaussian mixture distribution in the initial hidden Markov model; And the frequency of occurrence of a combination of a Gaussian mixture distribution having the same time and a Gaussian distribution in the Viterbi sequence of the Gaussian mixture distribution and the maximum likelihood sequence of the Gaussian distribution obtained by the first processing means. For all combinations of the Gaussian mixture distribution and the Gaussian distribution included in the initial hidden Markov model, when a frame error occurs in each Gaussian mixture distribution and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution of the combination Is calculated, and when each calculated frame error probability exceeds a predetermined threshold value, A second processing means for adding a Gaussian distribution as a new component of the Gaussian mixture distribution, wherein each Gaussian distribution added as a new component of each Gaussian mixture distribution by the second processing means is a Gaussian mixture. In the initial hidden Markov model, the distribution is a component shared by both the Gaussian mixture distribution belonging to the component and the Gaussian mixture distribution newly belonging to the component by the second processing means. . Therefore, the initial HMM
Is re-learned after adding and reconfiguring components as described above, compared to the initial HMM,
It is possible to enhance the expressive power of the distribution while maintaining the robustness as an acoustic model. Therefore, by performing voice recognition using the HMM, voice recognition can be performed at a higher voice recognition rate than in the related art. Further, the calculation time for determining the shared structure can be reduced to almost half as compared with the second conventional example, and the HMM can be constructed at a higher speed.

Further, in the speech recognition apparatus according to the present invention, the speech recognition is performed based on the speech signal of the uttered speech sentence using the acoustic model generated by the acoustic model generation apparatus according to claim 1 or 2. Voice recognition means for performing the operation. Therefore, as for the HMM obtained by reconstructing the initial HMM by adding components as described above and then re-learning, the expression power of the distribution can be enhanced while maintaining the robustness as an acoustic model, as compared with the initial HMM. . Therefore, by performing voice recognition using the HMM, voice recognition can be performed at a higher voice recognition rate than in the related art. Further, the calculation time for determining the shared structure can be reduced to almost half as compared with the second conventional example, and the HMM can be constructed at a higher speed.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice recognition device according to an embodiment of the present invention.

FIG. 2 is a timing chart showing processing of a word hypothesis narrowing section 6 in the voice recognition device of FIG.

FIG. 3 is a flowchart illustrating a first part of the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 of FIG. 1;

FIG. 4 is a flowchart illustrating a second part of the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 of FIG. 1;

FIG. 5 is a flowchart illustrating a third part of the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 of FIG. 1;

FIG. 6 is a flowchart illustrating a fourth part of the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 in FIG. 1;

FIG. 7 is a diagram illustrating an example of the configuration of a Viterbi sequence and a maximum likelihood sequence in the HMM reconfiguration processing performed by the HMM reconfiguration unit 22 in FIG. 1;

FIG. 8 is a diagram illustrating an example of calculation of a frame error probability in the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 in FIG. 1;

9 is a diagram illustrating addition of a component based on an error probability in the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 in FIG. 1;

FIG. 10 is a graph showing an example of the distribution of the number of mixtures after the HMM reconfiguration processing executed by the HMM reconfiguration unit 22 in FIG. 1;

11 is an experimental result of the voice recognition device of FIG. 1,
It is a graph which shows a comparison of a speech recognition result.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... microphone, 2 ... feature extraction part, 3, 5 ... buffer memory, 4 ... word collation part, 6 ... word hypothesis narrowing part, 7 ... likelihood correction part, 11 ... phoneme HMM memory, 12 ... word dictionary memory, 13: Statistical language model, 14: Underestimated likelihood memory, 21: Initial HMM generating unit, 22: HMM reconstructing unit, 23: Re-learning unit, 30: Voice data memory, 31: Initial HMM memory, 32: Re Configured HMM memory.

Claims (3)

[Claims] 1. A first generating means for generating an initial hidden Markov model by a predetermined learning algorithm based on a characteristic parameter of predetermined audio data, and an initial hidden Markov model is generated for the audio data. By adding a component of the Gaussian mixture distribution of the Hidden Markov Model based on the tendency of a frame error, which is an identification error of a frame unit at a predetermined time, the initial Hidden Markov Model generated by the first generation unit is obtained. A second generation unit configured to generate a reconstructed hidden Markov model, and a hidden Markov model generated by the second generation unit by a predetermined learning algorithm based on the feature parameter of the audio data. By retraining the model,
An acoustic model generation device, comprising: a third generation unit configured to generate an acoustic model that is a re-learned hidden Markov model. 2. The acoustic model generation device according to claim 1, wherein the second generation unit executes a Viterbi alignment process between the initial hidden Markov model and the audio data.
(A) the Viterbi sequence of the Gaussian mixture distribution included in the initial hidden Markov model, and (b) the highest likelihood is given to each frame of the audio data, and each Gaussian is included in the initial hidden Markov model. First processing means for obtaining a maximum likelihood sequence of a Gaussian distribution included as a component of a mixture distribution, and a Viterbi sequence of a Gaussian mixture distribution obtained by the first processing means, and a maximum likelihood sequence of a Gaussian distribution, Based on the Gaussian mixture distribution at the same time and the frequency of each of the combinations of the Gaussian distributions, for all combinations of the Gaussian mixture distribution and the Gaussian distribution included in the initial hidden Markov model, the frame in each Gaussian mixture distribution When an error occurs and the maximum likelihood Gaussian distribution at that time is the Gaussian distribution of the combination, the frame error probability A second processing means for calculating a rate and adding the Gaussian distribution as a new component of the Gaussian mixture distribution when each of the calculated frame error probabilities exceeds a predetermined threshold value. Each Gaussian distribution added as a new component of each Gaussian mixture distribution by the processing means of the above, the Gaussian distribution in the initial hidden Markov model,
An acoustic model generation apparatus characterized in that the component is a component shared by both the Gaussian mixture distribution belonging to the component and the Gaussian mixture distribution newly belonging to the component by the second processing means. 3. A voice recognition device comprising: a voice recognition unit configured to recognize a voice based on a voice signal of an input utterance voice sentence, using a voice model generated by the voice model generation device according to claim 1 or 2. Speech recognition device.