JPH064092A - Hmm generating device, hmm storage device, likelihood calculating device, and recognizing device - Google Patents

Abstract

PURPOSE:To obtain the recognizing device using HMM(Hidden Markov Model) which has high recognition precision and is small in calculation quantity. CONSTITUTION:The device is equipped with continuous probability distribution HMM generating means 101, 102, and 104 which generate a continuous probability distribution HMM, a clustering means 106 which clusters learnt vectors, and a generation degree calculating means 108 which calculates the generation degrees of respective clusters in respective states of the HMM from probability density functions in the respective states of the continuous type HMM; and the generation degrees are used as generation degrees in the respective states of the respective cluster levels to generate a discrete probability distribution HMM.

Description

Detailed Description of the Invention

[0001]

[Industrial application] HMM (Hidden Markov Mode) applicable to pattern recognition such as voice recognition
l)) A creation device, an HMM storage device, a likelihood calculation device, and a recognition device.

[0002]

2. Description of the Related Art An HMM is applicable to general time series signal processing, but for convenience of description, an example applied to speech recognition will be described below.

First, a voice recognition device using an HMM will be described.

FIG. 3 is a block diagram of a voice recognition device using an HMM. The voice analysis unit 201 uses a well-known method such as a filter bank, Fourier transform, or LPC analysis for the input voice signal to set a fixed time interval (called a frame), for example, 1
Convert to a feature vector every 0 msec. Therefore, the input speech signal is a sequence of feature vectors Y = (y (1), y (2), ..., Y
(T)). T is the number of frames. The codebook 202 holds the labeled representative vector. The vector quantization unit 203 replaces each vector of the vector series Y with a label corresponding to the representative vector registered in the codebook 202 that is closest to the vector. The HMM creating unit 204
The HMM corresponding to each word which is a recognition vocabulary is created from the training data. That is, in order to create the HMM corresponding to the word v, first, the structure of the HMM (the number of states and the transition rules allowed between those states) is appropriately determined, and then the word v is repeated many times as described above. From the label sequences obtained by uttering, the state transition probabilities in the model and the label occurrence probabilities associated with the state transitions are calculated so that the occurrence probabilities of the label sequences are as high as possible. The HMM storage unit 205 stores the H thus obtained.
The MM is stored for each word. Likelihood calculator 20
6 calculates the likelihood of the label sequence of each model stored in the HMM storage unit 205 with respect to the label sequence of the unknown input speech to be recognized. The comparison determination unit 207 determines a word corresponding to the model that gives the maximum value of the likelihood of each model obtained by the likelihood calculation unit 206 as a recognition result.

The recognition by the HMM is specifically performed as follows. That is, the label sequence obtained for the unknown input is O = (o (1), o (2), ..., O (T)), the model corresponding to the word v is λ ^v , and the model λ ^v , An arbitrary state sequence of length T generated by X = (x (1), x (2), ...
, X (T)), the likelihood of λ ^{v for} the label sequence O is [exact solution]

[0006]

[Equation 1]

[Approximate Solution]

[0008]

[Equation 2]

Or, taking the logarithm

[0010]

[Equation 3]

Is defined by Where P (x, y | λ ^v )
Is the joint probability of x and y in the model λ ^v .

Therefore, if, for example, (Equation 1) is used,

[0013]

[Equation 4]

Then, v ^ is the recognition result. The same applies when using (Equation 2) and (Equation 3).

P (O, X | λ) is obtained as follows.

Now, for the state q _i (i = 1 to I) of the HMMλ, for each state q _i , the occurrence probability b _i (o) of the label o and the state q _i (i = 1 to I) When the transition probability a _ij to q _j (j = 1 to I + 1) is given, the state sequence X = (x
(1), x (2), ..., x (T + 1)) and label series O = (o
(1), o (2), ..., o (T))

[0017]

[Equation 5]

It can be defined as Where π _{x (1)} is the state x (1)
Is the initial probability of. It is also assumed that x (T + 1) = I + 1 is the final state and no label is generated.

In this example, the input feature vector y is converted into a label, but there is also a method of using the feature vector y as it is in place of the label occurrence probability in each state and providing the probability density function of the feature vector y in each state. . At this time, the occurrence probability b _i (o) in the state q _i of the label o in (Equation 5) B _i when instead would use a probability density b _i (y) of the feature vector y in (hereinafter, the probability z is generated in b _i (z) is z state i when the label, z is a vector of (z) means the probability density of z). At this time, the above (Formula 1), (Formula 2), and (Formula 3) are as follows. [Exact solution]

[0020]

[Equation 6]

[Approximate Solution]

[0022]

[Equation 7]

Alternatively, the following equation can be obtained by taking the logarithm.

[0024]

[Equation 8]

The above, final recognition results in the use of any method, if preparing a HMMramuda ^v for v = 1 to V for each word v, the input speech signal Y
Against

[0026]

[Equation 9]

Or

[0028]

[Equation 10]

Is the recognition result of Y. Of course, Y here
Is the input label sequence according to the above method,
A feature vector series or the like.

[0030]

In such a conventional example, the one that transforms the input feature vector into a label is called a discrete probability distribution HMM, and the one that uses the input feature vector as it is is called a continuous probability distribution HMM. At this time, the characteristics of both of them are as follows.

In the discrete probability distribution HMM, the occurrence degree b _i (C _m ) of each label in each state in the calculation of the likelihood of the model with respect to the input label sequence is stored from the storage device stored in advance in association with the label. Although there is an advantage that the amount of calculation is very small because it can be executed by reading out, there is a problem that recognition accuracy deteriorates due to an error associated with quantization. To avoid this, it is necessary to increase the number of labels (the number of clusters), but as the number of labels increases, the number of learning patterns necessary for learning the model becomes enormous. Here, when the number of learning patterns is insufficient, the b _i (C _m ) may frequently become 0, and correct estimation cannot be performed. For example, the following occurs.

The codebook is created by converting the uttered voices of a large number of speakers into a feature vector sequence for all the words to be recognized, clustering this feature vector set,
This is done by labeling each cluster. Each cluster has a representative vector of that cluster, called the centroid, which is usually the expected value of the vector classified into each cluster. The codebook stores these centroids in a form searchable by the label.

Now, consider a case where, for example, there is a word "Osaka" in the recognition vocabulary, and a model corresponding to this is made. A voice sample corresponding to the word "Osaka" uttered by a large number of speakers is converted into a feature vector sequence, each feature vector is compared with the centroid, and a label corresponding to the nearest centroid is a quantum of the feature vector. Will be In this way, the "Osaka"
Each voice sample for is converted into a label sequence. A model corresponding to the word "Osaka" is created by estimating the parameters of the HMM from the obtained label series so that the likelihood for the label series is maximized. The well-known Baum-Welch method or the like can be used for this estimation.

In this case, among the labels in the codebook, some may not be included in the learning label series corresponding to the word "Osaka". The occurrence probability of the label not included is estimated to be "0" in the learning process. Therefore, it is sufficient that there is a label that is not included in the label sequence used to create the model of "Osaka" in the label sequence in which the word "Osaka" converted at the time of recognition is converted. It is possible. In this case, the probability that the “Osaka” label sequence uttered at the time of recognition will occur from the “Osaka” model will be “0”. However, even in such a case, even if the label is different, at the stage of the feature vector before being converted to the label, it is quite close to the speech sample used for learning the model, and if you look at the stage of the vector, it is sufficient to say “Osaka”. It may be recognized that It is quite possible that a label will be converted to a completely different label with a slight difference, even though they are similar at the vector level because they originally speak the same word. It can be easily imagined that such a situation adversely affects the recognition accuracy. Such problems occur more frequently as the number of clusters increases and the number of training data decreases.

In order to eliminate this problem, it is necessary to take measures such as smoothing and complementing labels that do not appear (are not included) in the training set. Starting from the idea of reducing the number of parameters using the concept called “conclusion”, when the 0 probability is estimated, it is replaced with a small amount instead of 0, or the boundary of the cluster such as fuzzy vector quantization. Although various methods for smoothing and complementing, such as a method for blurring, have been proposed, none of these methods fundamentally solve the above problems. Also, there are factors that must be determined empirically depending on the case, and there is no theoretical indicator that determines those factors.

On the other hand, in the continuous probability distribution HMM, the distribution shape is given in advance in the form of a function such as a normal distribution and the parameters defining this function are estimated from the learning data. Therefore, the number of parameters to be estimated is small, the parameters can be accurately estimated with less learning patterns than the discrete type, there is no need to consider smoothing and complementation, and generally a higher recognition rate than the discrete type can be obtained. It is reported that

The number of parameters in the four-state, three-loop HMM as shown in FIG. 4 is compared between the discrete type and the continuous type, for example, as follows. In the case of the discrete type, if the type of label used is 256, the label occurrence probability is 2
56 × 3 = 768, the transition probability is 6, and a total of 874 are required per model. In the case of the continuous type, if the 10-dimensional normal distribution is used, the average vector is 10 × 3 = 30, the variance / covariance matrix is 55 × 3 = 165 (∵symmetric matrix), and the transition probability is 6, totaling 201. The value of the power parameter in the continuous type is 1/4 or less of that in the discrete type.

However, although the continuous type is excellent in recognition accuracy, there is a problem that the amount of calculation becomes much larger than that of the discrete type. That is, assuming that the input feature vector y (t) has a normal distribution of the mean vector μ _i and the variance-covariance matrix Σ _{i in} the state i, the calculation of the occurrence probability (density) of y (t) in the state i (y (t) −μ _i ) ^T Σ _i ⁻¹ (y (t) −μ _i ), which is necessary. For example, in a 10-dimensional continuous HMM, this calculation alone requires 110 multiplications. And
This is (number of states x number of input frames) times this for one model. Therefore, if the above model is assumed when the number of input frames is 50, (y (t) −μ _i ) ^T Σ _i ⁻¹ (y (t) −μ _i ) of one model is required. The number of multiplications in the calculation is 110 × 3 × 50 = 16500
When the number of words is 500, this is further multiplied by 500. That is, in that case, only the multiplication of this part is 82
It requires 50,000 times.

In the case of the discrete type, once the vector quantization calculation is completed, it is only necessary to read the occurrence probability of the label from the storage device according to the label as described above. See y
The calculation required for vector quantization of (t) is the calculation of the distance or the similarity between 256 representative vectors and y (t) in the above example. If the distance is (Euclidean distance) ² , the calculation required to label y (t) is 25 times 10 subtractions, 10 multiplications, and 10 additions.
6 times. Therefore, in 50 frames, if only multiplication is considered, 10 × 256 × 50 = 1280 thousand times. If the vector quantization is performed by a method called binary search, the above 256 is 2log ₂ 256 = 16.
In other words, 10 × 16 × 50 = 8000 times.

As described above, when the discrete type is used, the amount of calculation is remarkably reduced, and when the number of recognized words is increased in the continuous type, the amount of calculation is also increased in proportion thereto. This calculation is necessary only when the signal is vector-quantized once, and this calculation amount does not change even if the number of recognized words increases.

In short, the discrete type has a small calculation amount but has a problem in recognition accuracy, and the continuous type has a good recognition accuracy but has a problem in the calculation amount.

In consideration of such problems of the conventional HMM, the present invention provides an HMM creating device, an HMM storage device, a likelihood calculating device, and a recognizing device which have high recognition accuracy and can reduce the amount of calculation. With the goal.

[0043]

According to the present invention, there is provided a continuous probability density distribution HMM creating means, a clustering means for clustering a training vector set and labeling each cluster, and each cluster in each state of the HMM. The degree of occurrence of the label is calculated based on the training vector included in each cluster and the continuous probability density distribution HMM.
And a label occurrence degree calculating means for calculating from a probability density function in each state of HMM, and obtaining a label occurrence degree in each state of the HMM as an output of the label occurrence degree calculating means to create a discrete probability distribution HMM. It is a device.

[0044]

In the present invention, the probability density function in each state of the HMM is obtained by the continuous probability distribution HMM creating means, the training vector set is clustered by the clustering means, and a label is given to each cluster, and the label occurrence degree calculating means. Therefore, the degree of occurrence of each of the clusters in each state of the HMM, that is, the label, is determined by the training vector included in each cluster and the continuous probability density distribution H.
A discrete probability distribution HMM is created by calculating from the probability density function in each state of MM.

[0045]

Embodiments of the present invention will be described below with reference to the drawings.

First, the definitions of the symbols used hereinafter will be summarized. At that time, for the sake of simplicity, unless misunderstanding occurs,
The states q _i , q _j, etc. will be simply referred to as i, j, etc.
In addition, the case where the model learning is performed for the word v will be described, and when it is necessary to distinguish them, the subscript v is added to the right shoulder of the parameter, and this is usually omitted. It is as follows.

I = 1, 2, ..., I + 1: i-th state [a _ij ]: transition matrix a _ij : transition probability from state i to state j r: training pattern number for word v (r = 1, ..., R) y ^(r) (t): observation vector in the t-th frame of the training pattern r o ^(r) (t): observation label b _i (y ^{(r (r)} in the t-th frame of the training pattern ^{r )} (t)): probability density b _i (o ^(r) (t)) of observation vector y ^(r) (t) of frame t of training pattern r in observation state o of frame t of training pattern r ^(r) (t) occurrence degree in state i (probability, probability density, etc.) y ^(r) = (y ^(r) (1), y ^(r) (2), ..., y ^(r) (T ^(r) )): vector sequence of training pattern r (where r = 1, 2, ...,
R) O ^(r) = (o ^(r) (1), o ^(r) (2), ..., O ^(r) (T ^(r) )): r-th label series for word v ( However, r = 1, 2, ...
.., R) X ^(r) = (x ^(r) (1), x ^(r) (2), ..., x ^(r) (T ^(r) ), x ^(r)
(T ^(r) +1)): State sequence corresponding to X ^(r) or O ^(r) x ^(r) (t): State at frame t of the r-th training pattern for word v T ^(r) : words v the r th training pattern frame number mu _i of for: b _i (y) mean vector sigma _i of: b _i (y) of the covariance matrix xi] _i: defining a probability distribution of the observed vectors in state i Set of parameters (ξ _i = {μ _i , Σ _i }) λ _i = [ξ _i , {a _ij } _{j = 1, ..., I + 1} ]: Set of parameters of state i λ = {λ _i }: Set of all parameters (a model having λ as a parameter is also referred to as model λ) P (Y | λ): Probability that observation vector series Y occurs from model λ Density P (O | λ): Probability that observation label sequence O is generated from model λ π _i : Probability that state i occurs when t = 1 First, a method of learning a continuous probability distribution HMM corresponding to word v will be described.

The problem is that r = 1 prepared for word v.
~ R training patterns P (Y ⁽¹⁾ , Y ⁽²⁾ ,
^···, Y ^(R) | is to estimate the parameters lambda to maximize lambda).

If Y ^(r) are independent of each other,

[0050]

[Equation 11]

Is given by Where the following auxiliary function Q
Define (λ, λ ').

[0052]

[Equation 12]

At this time, the following can be said. Q (λ, λ ')
If ≧ Q (λ, λ), P (Y ⁽¹⁾ , ..., Y ^(R) | λ ') ≧ P (Y ⁽¹⁾ ,
, Y ^(R) | λ), and the equal sign holds when λ ′ = λ. Therefore,

[0054]

[Equation 13]

If it is possible to obtain, by repeatedly applying (Equation 13) with λ ^* → λ, λ becomes P
(Y ⁽¹⁾ , ..., Y ^(R) | λ) stop point, that is, P (Y ⁽¹⁾ , ..., Y
The maximum value of ^(R) | λ) or the point that gives the saddle point is converged, and the change rate of P (Y ⁽¹⁾ , ..., Y ^(R) | λ) becomes equal to or less than a predetermined threshold value. A local optimum solution can be obtained by repeating the operation.

Next, a method of estimating parameters using Q (λ, λ ') will be described.

By modifying (Equation 12), the following equation is obtained.

[0058]

[Equation 14]

From the above description, if Q (λ, λ ') is regarded as a function of λ', and λ'where Q (λ, λ ')> Q (λ, λ) is found,
It becomes an updated version of λ, and P (Y ⁽¹⁾ , ..., Y ^(R)
| λ) is a constant value for λ ', so remove this

[0060]

[Equation 30]

Then, it is similar to finding λ ′ such that Q ′ (λ, λ ′)> Q ′ (λ, λ). However, here

[0062]

[Equation 15]

It is said that.

(Equation 14) becomes as follows.

[0065]

[Equation 16]

[0066] Re-estimate of when maximized for the first term π _i 'π _i π _i ^* is

[0067]

[Equation 17]

[0068] Re-estimate of when maximized for a _ij 'from the second term on the right side a _ij a _ij ^* is

[0069]

[Equation 18]

[0070] From the third term on the right side μ _i ', Σ _i' be maximized for, mu _i, re-estimated value of each ^{_{Σ i μ i *, Σ i}} * is

[0071]

[Formula 19]

[0072]

[Equation 20]

Here, ξ ^(r) _ij (t) is calculated as follows. That is,

[0074]

[Equation 21]

In summary,

[0076]

[Equation 22]

It is

At this time

[0079]

[Equation 23]

[0080]

[Equation 24]

The following recurrence formula holds. Therefore, α
^(r) ₁ (1) = 1 and a proper initial value is given to the parameter λ, and α ^(r) _j (t) is calculated according to ⁽ Equation 23) for t = 1 to T ^(r) +1 and j = 1 to I + 1. , Β ^(r) _{I + 1} (T ^(r) +1) =
If t = T ^(r) +1 to 1 and i = I to 1 as 1, and β ^(r) _i (t) is sequentially calculated according to (Equation 24), (Equation 15) can be calculated.

The actual calculation procedure for parameter estimation is as follows.

(1) L ₁ = ∞ (2) For i, j = 1 to I λ _i = {(a _ij ) _{j = 1, ..., I + 1} , μ
Give an appropriate initial value to _i , Σ _i }.

(3) r = 1 to R, t = 2 to T ^(r) , i =
For 1 to I + 1, α ^(r) _i (t) is calculated according to (Equation 23) with λ = {λ _i }.

(4) r = 1 to R, t = 2 to T ^(r) , i =
^Let β ^(r) _i (t) and ξ ^(r) _ij (t) for 1 to I + 1 be λ = {λ
_i } is calculated according to (Equation 24) and (Equation 22), respectively.

(5) For r = 1 to R, i, j = 1 to I + 1, the numerator of (Equation 18), (Equation 19), (Equation 20): a _{ij, num} (r), μ _{i, num} (r), Σ _{i, num} (r) and denominator: Den _i (r) = a _{ij, denom} (r) = μ _{i, denom} (r) = Σ
Calculates _{i, denom} (r).

[0087] _{(6) a ij, μ i} , re-estimated value of Σ _i a _ij ^*, μ _i
^* , Σ _i ^* is calculated according to the following (number).

[0088]

[Equation 25]

(7) For i, j = 1 to I + 1, a _ij =
The re-estimated parameter set λ = {λ _i } is obtained by performing the substitution a _ij ^* , μ _i = μ _i ^* , Σ _i = Σ _i ^* .

(8) r = 1 to R, t = 2 to T ^(r) , i =
Parameter set λ obtained in step (7) for 1 to I + 1
Against

[0091]

[Equation 26]

Calculate

(9) | L ₁ −L ₂ | / L ₁ > ε, then L ₂ =
Leave L ₁ and go to step (4), otherwise end.

Ε in the step (9) is an appropriately small positive number that determines the width of convergence, and its value is selected as a practical value depending on the situation.

As described above, the continuous probability distribution HMM is obtained. The present invention is based on this, but the discrete probability distribution HMM is obtained.
The MM is obtained by the following procedure.

(1) Clustering of learning vectors is performed to calculate M clusters. The cluster name is C ₁ , C ₂ ,
..., C _m , ..., C _M. Let the training vectors belonging to the cluster C _m be y _m (1), y _m (2), ..., Y _m (K ^m ).

(2) The HMM using the continuous HMM
The degree of occurrence of C _m (m = 1, ..., M) in each state is calculated.

Here, various methods of defining the degree of occurrence of each label can be considered. That is, (a) the probability density of occurrence of a centroid of C _m in state i, (b) the average or median of the probability densities of learning vectors belonging to C _m , and (a) and (b) the sum of those clusters. Is normalized so that it becomes 1, and in the case of the average value in (b), the average thereof may be an arithmetic average, a geometric average, a harmonic average, or the like. Here, as an embodiment of the present invention, the method (b) will be described by using an arithmetic mean as an example and not normalizing. The b _i (y) used in the following equation is obtained from the estimated parameters of the continuous HMM. In this case, the occurrence degree b _im of the cluster C _m in the state i is given by the following equation.

[0099]

[Equation 27]

As the clustering method in the step (1), for example, a well-known method called LBG method can be used (the description of the specific method is omitted). As the data to be clustered, it is possible to use the entire set of feature vectors forming a pattern corresponding to the word speech of v = 1 to V used for learning of the HMM.

1 and 2 show an embodiment of the HMM creating apparatus of the present invention. The structure and operation will be described below with reference to the drawings.

The feature extraction unit 101 uses the well-known method to convert the speech signals of the training words r = 1 to R ^v prepared for model creation corresponding to the word v (= 1, ..., V) into feature vector vectors. series

[0103]

[Equation 28]

Convert to

The word pattern storage unit 102 has RAM, R
A means such as an OM and various disks, which stores R ^v learning words for creating the model λ ^v in the form of the feature vector series.

The buffer memory 103 retrieves R ^v word patterns for v stored in the word pattern storage unit 102 and temporarily stores them.

The parameter estimation unit 104 uses the model λ
^v perform the step (1) to (9) to create a word v
Estimate the model λ ^v corresponding to.

The first parameter storage unit 105 temporarily stores the re-estimated value of the parameter obtained in the step (6). The parameter estimation unit 104 re-estimates using the value of the parameter storage unit 105.

The clustering unit 106 is stored in the word pattern storage unit 102.

[0110]

[Equation 29]

Cluster feature vector sets into M clusters. At this time, the label of the m-th cluster is C _m and the centroid is y _0m .

The cluster vector storage unit 107 stores the vector and centroid of each of the M clusters obtained by the clustering unit 106 in a form that can be referred to by m.

The label generation degree calculation unit 108 calculates the vector y _m (1) of the cluster C _m stored in the cluster vector storage unit 107 from the probability density function of the model λ ^v stored in the parameter storage unit 105. The probability density of y _m (K ^m ) is v = 1, ..., V, i = 1, ..., I, m = 1 ,.
Compute for M, and according to (Equation 27), the HM of the word v
Calculate the occurrence degree b ^v _im of C _m in the state i of M.

The second parameter storage unit 109 stores the word v =
For storing parameters corresponding to 1 to V,
The parameters corresponding to the respective words v = 1, ..., V are stored in the parameter storage units 1 ,. That is, the transition probability corresponding to each state of each word is read from the first parameter storage unit 105 and stored in a form that can be referred to by v, i, j.
Further, the degree of label occurrence in each state of each word is read from the label occurrence degree calculator 108, and v,
It is stored in a form that can be referred to by i and m.

As described above, the set of vectors forming the pattern set used for learning is clustered, and the occurrence degree b _im of the vector included in the cluster m in the state i of the HMM is obtained as a continuous probability distribution type HMM. The obtained probability density is used for conversion into a discrete probability distribution type HMM.

Next, the structure and operation of an apparatus for recognizing an actual input voice using the above model will be described at the same time.

FIG. 5 is a block diagram of the recognition device.

The feature extraction unit 401 is the feature extraction unit 1 of FIG.
01 has exactly the same configuration and function.

The codebook 403 stores the centroid of each cluster stored in the cluster vector storage unit 107 of the HMM creating apparatus of FIGS. 1 and 2.

The vector quantizer 402 includes a feature extractor 4
01 output feature vector y (t) and codebook 403
The distance from the representative vector y _0m (m = 1, ..., M) of each of the clusters stored in
The feature vector series is converted into a label series by replacing it with the label of the cluster corresponding to the representative vector closest to (t).

The parameter storage unit 404 has exactly the same configuration and function as the parameter storage unit 109 of FIG. 2, and the parameter storage unit v stores the words v (= 1, ...,
The parameters of the model corresponding to V) are stored.

Likelihood calculation section 405 includes vector quantization section 4
The likelihood of each model with respect to the label sequence obtained as the output of 02 is calculated using the contents of the parameter storage unit 404. That is, in the likelihood calculation unit v, the parameter storage unit v
Content is used. The likelihood calculation method is (Equation 1),
Either (Equation 2), (Equation 3), or the like may be used.

The comparison determination unit 406 compares and determines which output of the likelihood calculation units 1, ..., V included in the likelihood calculation unit 405 is the maximum, and the word corresponding to that is determined as the recognition result. This is output, and the calculation corresponding to (Equation 4) is executed.

The result of word recognition is output from the comparison / determination unit 406.

In the present embodiment, it is described that a word is recognized. However, in the present invention, the word may be replaced with a phoneme, a syllable, or the like, and may be applied to a pattern other than voice. .

Further, in the present embodiment, the distribution of the feature vector is described as following a single normal distribution in each state, but in the present invention, a more precise label generation degree is obtained by using a so-called mixed distribution. Of course, it is possible.

Further, the present invention is applicable not only to the voice recognition device but also to other time series signal processing fields.

Each means of the present invention may be realized by software using a computer or by using a dedicated hardware circuit having each of these functions.

[0129]

As is apparent from the above description,
The present invention relates to an HMM creating means for creating a continuous probability density distribution HMM, a clustering means for clustering a training vector set and giving a label to each cluster, and H
The label generation degree calculating means for calculating the degree of occurrence of each label according to each cluster in each state of MM from the training vector included in each cluster and the probability density function in each state of the continuous probability density distribution HMM is provided. So
It is possible to solve the problem of the discrete HMM, that is, the estimation error due to the lack of training data and its bias, and to realize a model that takes advantage of the advantage that the discrete HMM has a small amount of calculation.

[Brief description of drawings]

FIG. 1 is a part of a block diagram showing an embodiment of an apparatus for estimating parameters of an HMM according to the present invention.

FIG. 2 is the rest of the block diagram showing one embodiment of the apparatus for estimating the parameters of the HMM according to the present invention.

FIG. 3 is a block diagram illustrating a conventional example of a voice recognition device using an HMM.

FIG. 4 is a configuration diagram of an HMM showing a configuration of a continuous probability distribution type HMM.

FIG. 5 is a block diagram showing an embodiment of a voice recognition device using an HMM constructed according to the present invention.

[Explanation of symbols]

 101 ... Feature extraction unit, 102 ... Word pattern storage unit, 103 ... Buffer memory, 104 ... Parameter estimation unit 105 ... Parameter storage unit 106 ... Clustering unit 107 ... Cluster vector storage unit 108 ... Label generation degree calculation unit 109 ... Parameter storage unit

Claims (7)

[Claims] 1. An HM for creating a continuous probability density distribution HMM.
M creating means, clustering means for clustering a training vector set and giving a label to each cluster, and the degree of occurrence of each label according to each cluster in each state of the HMM, the training included in each cluster An HMM creation device comprising a vector and a label generation degree calculation means for calculating from a probability density function in each state of the continuous probability density distribution HMM. 2. A state transition probability storing means for storing the state transition probability obtained by the HMM creating apparatus according to claim 1, and a label occurrence degree storing means for storing the occurrence degree of each label in each state. HMM characterized by
Storage device. 3. The vector series by calculating which cluster of claim 1 each vector of the feature vector series constituting the input pattern belongs to and replacing each vector with the label of the cluster to which the vector belongs. Is described as a parameter stored in the HMM storage device based on the vector quantization means for converting the label into a label sequence, the state transition probability stored in the HMM storage device according to claim 2, and the degree of label occurrence in each state. Of the HMM,
A likelihood calculating device, comprising: a likelihood calculating unit that calculates a likelihood for the input pattern. 4. The likelihood calculation device according to claim 3 is provided for each recognition unit, the likelihood of each recognition unit model for an input signal is calculated, and the input signal is recognized as the recognition signal from the value of the likelihood. A recognition device characterized by determining which of the units. 5. The label generation degree calculation means includes, when the cluster is C _m (m = 1, ..., M), from the probability density function of the state i of the continuous probability density distribution HMM to C _m . The probability density of each of the training vectors to be obtained, and a characteristic value calculating means for calculating a characteristic value such as an average value or a median of the probability density is included, and the characteristic value is defined as the occurrence degree b _im of C _{m in} the state i. The HMM creating apparatus according to claim 1, wherein: 6. The label generation degree calculating means represents C _m from the probability density function of the state i of the continuous probability density distribution HMM when the cluster is C _m (m = 1, ..., M). 2. The probability density of the vector is calculated, and the probability density is used as the occurrence degree C _im of C _{m in} the state i.
The described HMM creation device. 7. A label occurrence rate calculating means is further from the _{b im, b im '= b} im / (b i1 + ··· + b iM) comprises generating the degree normalizing means for calculating, the normalized occurrence 7. The HMM creating apparatus according to claim 5, wherein the degree b _im ′ is the degree of occurrence of C _m in state i.