JPH08248986A - Pattern recognition method - Google Patents

Abstract

PURPOSE: To effectively reduce the number of model parameters without degrading the recognition performance by reducing the number of the total parameters of the entire model through the use of one common parameter for similar characteristic parameters. CONSTITUTION: Against the inputted vectors, the likelihood of the hidden Markov model (HMM), in which the output probability distribution of each state is expressed by a multi-dimensional continuous distribution, is computed and the method outputs the category, which expresses the model having a highest likelihood, as the recognition result. Moreover, when it is expressed by a single continuous probability distribution or a mixed continuous probability distribution in each state of the HMM, one dimensional continuous distribution existing in each dimension of the multi-dimensional continuous distribution, which constitutes the above distributions, has common parameters, which express the distributions, between the distribution that exists in each dimension of other multi-dimensional continuous distributions. Furthermore, the method is provided with the HMM in which the commonality relationship of these parameters is specified in an individual manner. Thus, the parameters are effectively learned and the computation cost of the output probability is reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a Hidden Markov Model (hereinafter referred to as HM).
(Hereinafter referred to as M), the likelihood of each model with respect to the input vector is obtained, and the input vector is recognized.

[0002]

2. Description of the Related Art H modeling based on probability and statistics
The MM method is a useful technique in pattern recognition of voices, characters, figures and the like. In the following, a conventional technique using the HMM method will be described taking speech recognition as an example. In a conventional voice recognition device, a method of modeling a voice to be recognized by using an HMM has high performance and has become the mainstream at present. The details of this HMM method are shown in, for example, Document 1 (Seiji Nakagawa: Speech recognition by probabilistic model, Institute of Electronics, Information and Communication Engineers). FIG. 2 shows a configuration example of a conventional voice recognition device using an HMM. The voice input from the input terminal 11 is A /
The D conversion unit 12 converts the digital signal. The voice feature parameter extraction unit 13 extracts voice feature parameters from the digital signal. An HMM created in advance for each speech unit to be recognized (for example, phoneme, syllable, word) is read from the HMM memory 14, and the model likelihood calculator 15 calculates the likelihood of each model for the input speech. The recognition result output unit 16 outputs the speech unit represented by the model having the largest likelihood as the recognition result.

FIG. 3A shows an example of a three-state HMM. Such a model is created for each voice unit (category).
Statistical distributions D1 to D3 of voice characteristic parameters are given to the states S1 to S3, respectively. For example, if this is a phoneme model, the first state represents the statistical distribution of the feature amount near the beginning of the phoneme, the second state near the center, and the third state near the end.

The feature amount distribution of each state is often expressed using a plurality of continuous probability distributions (hereinafter referred to as mixed continuous distributions) in order to express a complicated distribution shape. Various distributions can be considered as the continuous probability distribution, but a normal distribution is often used. In addition, each normal distribution is often represented by a multidimensional uncorrelated normal distribution having the same number of dimensions as the feature amount. FIG. 3B shows an example of continuous mixed distribution. In this figure, the normal distribution N with mean value vector μ ₁ and variance value σ ₁
(Μ ₁ , σ ₁ ) and N (μ ₂ , σ ₂ ) and N (μ ₃ , σ ₃ )
It is a case where it is expressed by three normal distributions of. Input feature vector _Xt = ( _{xt, 1, xt, 2,} ... _{xt, P} ) ^{T at time} _t
The state s of the mixed continuous distribution HMM for (P is the total number of dimensions)
The output probability b _s (X _t ) of

[0005]

[Equation 1]

It is calculated as follows. Where W_k ^sState
Weight for the k-th multidimensional normal distribution k included in state s
Represents a coefficient. Probability density P for multidimensional normal distribution k_k
^s(X _t) Is

[0007]

[Equation 2]

It is calculated as follows. Where μ _k ^s is k
Mean vector for the th multidimensional normal distribution k, Σ _k
^s also represents the covariance matrix. If the covariance matrix is only diagonal components, that is, the diagonal covariance matrix, then P
The logarithmic value of _k ^s (X _t ) is

[0009]

(Equation 3)

Can be expressed as Here, μ _{k, i} ^s is _{the i-} th component of the mean value vector of the k-th multidimensional normal distribution of state s _, and σ _{k, i} ^s is the k-th multidimensional normal of state s It represents the i-th diagonal component (dispersion value) of the covariance matrix of the distribution. This calculation is performed on the feature vector at each time of the input speech for the model of the recognition candidate, and the recognition result is output based on the obtained log-likelihood.

[0011]

In order to improve the recognition performance, it is necessary to increase the expression capability of the acoustic model, and for this reason, it is necessary to increase the number of model parameters. An enormous amount of data is required to learn a large number of model parameters, but in reality, only limited data can be collected, so the number of model parameters cannot be increased unnecessarily. If a model including a large number of parameters is trained with a small amount of data, the model will be deeply dependent on the training data, and a recognition error will occur even for data slightly different from the training data at the time of recognition. However, if the number of model parameters is small, the expression ability is low, and sufficient recognition performance cannot be obtained. Thus, there is a trade-off between model accuracy and robustness during recognition.
There is a problem that it is necessary to represent a more accurate model with a smaller number of model parameters.

Further, in the pattern recognition device based on HMM, the calculation cost of the output probability of the equation (2) is the highest.
In a typical example of a speech recognizer, this calculation consumes 45% to 65% of the total speech recognition processing time. Although real-time processing is an important issue from the viewpoint of human interface, there is a problem that the current processing speed is not sufficiently satisfactory.

Therefore, an object of the present invention is to effectively reduce the number of model parameters and efficiently learn parameters even with the same amount of data, while maintaining recognition ability of the model, without deteriorating recognition performance. An object of the present invention is to provide a pattern recognition method that can be performed and has a low calculation cost of output probability and can perform real-time processing.

[0014]

According to the present invention, the HM
Even if parameters of distribution existing between different models or states of M have similar properties, one parameter is commonly used to reduce the total number of parameters of the entire model. For example, parameters (means) expressing (defining) the normal distribution of each dimension that constitutes the multidimensional normal probability distribution,
It is characterized in that the variance value) is shared with other similar normal distribution parameters.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The method of the present invention will be described by taking the case where the multidimensional continuous distribution is a normal distribution as an example. H in the system now
Multidimensional normal distributions A and B existing in two states with MM
Focusing on N (μ ₁ ^A , Σ ₁ ^A ), N (μ _{1 A}
^B (Σ, ₁ ^B ), N (μ, Σ), represents that the mean value vector is μ and the covariance matrix is a normal distribution of Σ. Assuming that the dimensionality of the feature parameter is 4 and the covariance matrix has only diagonal components, these distributions are N (μ ₁ ^A , Σ ₁ ^A ) = {N (μ _1,1 ^A , σ _1,1 ^A ),
N (μ _1,2 ^A , σ _1,2 ^A ), N (μ _1,3 ^A ,
σ _1,3 ^A ), N (μ _1,4 ^A , σ _1,4 ^A )} N (μ ₁ ^B , Σ ₁ ^B ) = {N (μ _1,1 ^B , σ _1,1 ^B ),
N (μ _1,2 ^B , σ _1,2 ^B ), N (μ _1,3 ^B ,
σ _1,3 ^B ), N (μ _1,4 ^B , σ _1,4 ^B )}. Here, each of the second-dimensional normal distributions, N (μ _1,2 ^A , Σ _1,2 ^A ) and N (μ _1,2 ^B , Σ
_{When 1,2} ^B ) are similar, share them and
One distribution N (μ _1,2 ^C , Σ _1,2 ^C ) is represented, and each multidimensional normal distribution is represented by N (μ ₁ ^A , Σ ₁ ^A ) = {N (μ _1,1 ^A , σ _{1 , 1} ^A ),
N (μ _1,2 ^C , σ _1,2 ^C ), N (μ _1,3 ^A ,
σ _1,3 ^A ), N (μ _1,4 ^A , σ _1,4 ^A )} N (μ ₁ ^B , Σ ₁ ^B ) = {N (μ _1,1 ^B , σ _1,1 ^B ),
N (μ _1,2 ^C , σ _1,2 ^C ), N (μ _1,3 ^B ,
σ _1,3 ^B ), N (μ _1,4 ^B , σ _1,4 ^B )}. This makes the total number of parameters μ and σ 1
I was able to reduce from 6 to 14.

Next, in the above example, the input vector at time t is
Cutle X_t= (X_{t, 1}, X_{t, 2}, X _{t, 3}, X_{t, 4}) To
When calculating the likelihood, the distribution of each dimension is shared.
State the benefits of being Likelihood for each model
In order to calculate, the calculation of formula (3) is required. beginning
Then, it is assumed that the likelihood of the distribution A is calculated. Next, the total of distribution B
However, since the distribution of the second dimension is shared,
There is no need to calculate, and the calculation result of the second dimension for distribution A
The fruit can be used. Shared like this
The calculation for the parameters can be calculated by either model.
Once done, use the calculated results in other models
It is possible to reduce the calculation amount.

In an actual system, for example, as shown in FIG. 1A.
The index table of element parameters like
Mari Normals of each dimension existing in multidimensional normal distributions A and B
Cloth, N (μ_1,1 ^A, Σ_1,1 ^A), N (μ_1,3 ^A, Σ_1,3
^A), N (μ_1,4 ^A, Σ_1,4 ^A), N (μ_1,1 ^B, Σ
_1,1 ^B), N (μ_1,3 ^B, Σ_1,3 ^B), N (μ_1,4 ^B,
σ _1,4 ^B), N (μ_1,2 ^C, Σ_1,2 ^C)
Cox I₁~ I₇, And each multidimensional distribution A, B
Respectively I₁~ I₇Described in. That is, N (μ₁
^A, Σ₁ ^A) = {I₁, I₇． I₂, I₃}, N (μ₁
^B, Σ₁ ^B) = {I _Four, I₇． I_Five, I₆}. Change
In addition, a calculation result buffer is provided as shown in FIG.
The parameter calculation result of the distribution of each dimension for the vector
Data I_iRefer to and store. Calculation result buffer is initial
For example, -1 is set in the state.

In calculating the input vector X _t ,
For example, the distribution A (N (μ ₁ ^A , Σ ₁ ^A )) is performed, and the first element index is I ₁ , so the storage of the calculation result buffer (FIG. (B)) for index I ₁ is −1. If it is -1, the distribution A is referred to by referring to the index table of the parameter of the element of FIG. 1A.
The parameters μ _1,1 ^A and σ _1,1 ^A are read out and calculated, and the result is stored in the index I ₁ of the calculation result buffer. The index I ₇ of the next element is similarly calculated, and the calculation result is stored in the index I ₇ of the calculation result buffer. The same applies hereinafter. When performing the calculation on the distribution B, for the second index I ₇ , the location of the computation result buffer stores the result executed at the time of the computation on the distribution A,
Since it is a value different from -1, this stored value is used.

The sharing of calculation results by such sharing of parameters can be performed regardless of the number of models, the number of states, the number of distributions in states, and the number of dimensions of feature parameters. next,
An example of connecting distribution parameters with different sharing relations for each dimension is described. In the feature vector of voice, the amount of information held by the parameter differs for each dimension. For example, the cepstrum, which is one of the feature quantities, is often expressed in about 16 dimensions, but mainly contains elements of lower dimensions, and thus contains a large amount of information. Therefore, for low-dimensional distributions, the number of parameters to be shared is reduced, the degree of freedom is increased, and the expression ability of the distribution is enhanced. On the other hand, for high-dimensional distributions, the parameters are actively shared, and similar distributions are shared. By such an operation, even if the total number of distributions is the same, a more efficient expression can be realized by arranging the number of distributions non-uniformly for each dimension. It is obvious that the distribution numbers may not be arranged non-uniformly for each dimension, and that the dimensions may have the same distribution number. A method of sharing a multidimensional normal distribution has been proposed, but in this case, in order to share a vector, each dimension has the same sharing relationship. Therefore, the number of distributions existing in each dimension after sharing is the same, and the dimension having a large amount of information and the dimension having a small amount of information are represented by the same number of distributions. Therefore, even if the models have the same number of distributions (when the number of distributions is the same, the calculation amount is also the same), the commonization method according to the present invention increases the number of distributions in a dimension having a large amount of information. A model with higher performance can be constructed by reducing the number of distributions of a dimension with a small amount of information. In other words, models with the same performance can be realized with a smaller number of distributions, and the amount of calculation can be reduced.

In a statistical method such as HMM, there is a close relationship between the number of parameters and the amount of learning data. When a large number of distributions are included in the model, the number of parameters to be estimated increases and a large amount of training data is required. When the amount of training data is small, the generality of the model is lost. In the present invention, the number of distributions is effectively reduced while leaving the information necessary for recognition, so that high recognition performance can be obtained with a small amount of learning data.

In the voice recognition, speaker adaptation is performed by using a voice uttered by a speaker to adapt an acoustic model created for an unspecified speaker so as to match the speaker. In the actual situation of speaker adaptation, only a small amount of adaptation voice data is often obtained. When model parameters are learned with a small amount of data, only some model parameters related to the training data can be adapted if the parameters are set independently in each model. According to the sharing of the present invention, when some model parameters are adapted, the parameters of other models sharing them can be adapted at the same time.

Next, examples will be described. For HMM sharing (1) Model level sharing where different phoneme environments share the same model (first layer) Proceedings 1-5-15 and KF Lee
Other "Large-vocabulary speaker-independent continuou
s speech recognition using HMM "Proceedings of 198
8 International Conference on Acoustics, Speech an
d Signal Processing, pp 123-126.), (2) State level sharing (second layer) where different models share the same state (eg Takami, et al. “Sequential State Division Method (S
Automatic generation of hidden Markov networks by SS) ",
Acoustical Society of Japan 1991 Autumn Research Presentation Presentation 2-5
-13), (3) Basis distribution level sharing (third layer) where different states share the same multidimensional normal distribution (eg, XD Huang "Unified technique for vector quan
tization and hidden Markov modeling using semi-con
tinuous models ", Proceedingsof 1989 International
Conference on Acoustics, Speech and Signal Process
ing, pp.639-642.), (4) Sharing of feature level sharing the same one-dimensional normal distribution (mean value and variance value) of different multidimensional normal distributions according to the present invention (fourth layer) There is.

Step 1: In order to realize the sharing of the first layer and the second layer, the sequential state division method (SSS) is adopted. The 450 state HMnet in which each state is expressed by a single normal distribution is created using the data of one speaker. Step 2: After each state of the model is mixed into two distributions, learning is performed with the data of a large number of speakers, and HMne for unspecified speakers is calculated.
Create t (2-layer shared model).

Step 3: To realize the sharing of multidimensional normal distribution levels, 700 distribution clusters are generated from all distributions (2 mixture × 450 states = 900 distributions) (in the experiment, 256 clusters are provided for reference). , 64 cases). The distance measure of distribution i and distribution j was defined as follows.

[0025]

[Equation 4]

Here, μ and σ represent an average value and a variance value, respectively. P is the total number of dimensions. The representative distribution of each cluster is shared by the distribution within the cluster. Therefore, the total number of distributions is from 900 to 70 while maintaining the state 2 mixture.
The number is reduced to 0 (3-layer shared model). Step 4: In sharing the feature level, each dimension is represented by a different number of distributions. Here, the average inter-distribution distance of 900 distributions existing in each dimension of the model obtained in step 2 is calculated, and the distribution number m _p (p is a dimension) of each dimension is calculated based on the distance ratio. Decided. The distance measure is the same as in equation (4).

[0027] Step 5: Based on the model obtained in the step 3, independently in each dimension to generate m _p pieces of distribution clusters, determining a share relationship (4 hierarchical sharing model). In addition,
The furthest neighbor method (Furthe) for all clustering
st neighbor method) was used. In addition, with the three-layer shared model and the four-layer shared model, re-learning was performed after the sharing relationship of distribution was determined.

The performance of each hierarchical model thus obtained and the calculation time at the time of recognition were evaluated by a phoneme recognition experiment and a word recognition experiment. In the experiment, 10 males with 5240 word sets of ATR important words and 216 word sets were used. Of these, eight were learning speakers and two were evaluation speakers. The model was trained using a set of evenly selected 10,480 words from the even number of 5240 words of the learning speaker and a 216 word set of all the learning speakers.
For the evaluation of the phoneme recognition experiment, 5 from the odd-numbered word set
Twenty-four words were arbitrarily selected and used. In the word recognition experiment, 1500 words were arbitrarily selected from the odd-numbered words of 5240 words as a recognition target vocabulary, and 200 words were actually recognized and evaluated. The number of phoneme categories is 26. The parameters are the 16th-order cepstrum, the 16th-order Δ cepstrum, and the Δpower (33 dimensions in total).

FIG. 4 shows an example of the number of distributions assigned to each dimension of the feature quantity in step 4. Figure 4 shows each dimension
Average 64 distributions (total: 64 x 33 dimensions = 2112 distribution)
Is the result when. It can be seen that both the cepstrum and the Δ cepstrum have a larger average inter-distribution distance for the components of lower dimensions, and more distributions are assigned.

Next, FIG. 5 shows the structure of each model, the average phoneme recognition rate, the average word recognition rate, and the average calculation time (CPU tim).
The ratio of e) is shown. All of the three-layer shared model and the four-layer shared model in FIG. 5 are shared while always maintaining the structure of 450 state 2 mixture (900 distribution). In the four-tier shared model, the distribution numbers in each dimension are non-uniformly assigned, and the numerical values in FIG. 5 represent the average distribution numbers. A mean value, a covariance value, and a distribution weighting factor are considered in the total number of parameters. The calculation time was expressed as a ratio, with the CPUtime spent in the output probability calculation in word recognition being 1.0 for the two-layer shared model. Calculator is SUN SPA
RC10. For reference, the results of the phoneme environment independent model with three states and two mixtures are also shown.

In the three-layer shared model, the number of basis distributions is 70
Even if the number is reduced to 0, the performance does not change and the calculation time can be shortened. However, in this case, it is assumed that it is similar to the average value and the covariance value for all of the dimensions. , It is originally 9 when the number is reduced to 64.
An unreasonable sharing relationship is established for the 00 distributions, and the performance (recognition rate) sharply decreases as shown in FIG. In contrast,
In the four-layer shared model of the present invention, distributions are effectively assigned to each dimension, and the combination thereof represents 700 base distributions, and these represent 900 distributions. By these layering, efficient expression is made,
Even if the number of distributions is 256, the phoneme recognition rate is 87.6 and the word recognition rate is 90.0, which is the same result as the number of distributions 700 of the three-layer shared model. Even if the number of distributions is reduced to 64, the phoneme recognition rate is 86. .4, the word recognition rate is only 89.3, which is a slight decrease, and a recognition rate significantly higher than the distribution number 64 of the three-layer shared model is obtained, and the calculation time can be reduced.

Such an effect is obtained because the normal distribution is shared in each dimension independently, and as a result, as shown in FIG. 4, the number of distributions varies depending on the information amount of the dimension. In other words, high dimensions with a small amount of information are commonly shared, and the number of distributions is small, and low dimensions with a large amount of information are less shared, the number of distributions is large, and the number of distributions as a whole can be expressed efficiently with the model Because you can

The present invention can be used not only for voice recognition but also for any pattern recognition using HMM, such as character recognition using HMM and figure recognition.

[0034]

The effect of sharing model parameters according to the present invention has two points. One is that the learning efficiency of the model can be improved, and the other is that the amount of calculation at the time of recognition can be reduced. In general, model parameters are set independently for each model and learned using data in each category. However, if parameters with similar properties are shared between different models, the shared parameters can be learned by using data in both categories, so the apparent data amount will increase.

For example, since a phoneme environment-dependent model has a large number of multidimensional normal distributions (for example, 1000 or more), each dimension has the same number of one-dimensional normal distributions. For example,
Even when the normal distributions are merged from 1000 to m in each dimension, the number of multidimensional normal distributions that can be expressed is m ^P because the multidimensional normal distribution is expressed by the combination of m normal distributions. Yes (P is the number of dimensions), it retains considerable expressive power even after sharing.

Next, advantages will be considered from the viewpoint of calculation amount. Since many current HMMs assume a multidimensional uncorrelated normal distribution, the log-likelihood is calculated as in equation (3).
Equation (3) is the sum of the probability density values for the normal distribution in each dimension, and there is no calculation across dimensions. Therefore, the calculation can be considered independently in each dimension. Sharing the distribution has the advantage that the results of each dimension of equation (3) can be shared between the models and the amount of calculation at the time of recognition can be reduced.

It has been proposed that only the average value in each distribution be made common, but for example, consider the case where there are 1000 multidimensional normal distributions having 33-dimensional feature values in total (actually in speech recognition. Value). The number of parameters is 2 (average value, variance value) × 33 × 1000 = 66000.
If this is shared by only 64 average values, 33 × 10
64 = 35112. If the average value and the variance value are shared, the number will be 33 × 64 = 1112, which can be greatly reduced. As for the amount of calculation, the division can be further omitted once compared to the case where only the average value is shared.

[Brief description of drawings]

FIG. 1A is a diagram showing an example of a multidimensional normal distribution element index table, and B is a diagram showing an example of a calculation result buffer table.

FIG. 2 is a block diagram showing a general configuration of a pattern recognition device using an HMM.

3A is a diagram showing an example of an HMM, and FIG. 3B is a diagram showing an example of a mixture distribution.

FIG. 4 is a diagram showing an experimental example of the number of distributions assigned to each dimension of the feature amount in the present invention.

FIG. 5 is a diagram showing experimental results of recognition performance and calculation time of the conventional sharing method and the sharing method according to the present invention.

Claims (4)

[Claims] 1. The likelihood of a hidden Markov model in which the output probability distribution of each state is represented by a multidimensional continuous distribution is calculated for an input vector, and the category represented by the model with the highest likelihood is output as a recognition result. In the pattern recognition method described above, when each state of the hidden Markov model is represented by a single continuous probability distribution or a mixed continuous probability distribution, one-dimensional continuous existing in each dimension of the multidimensional continuous distribution that constitutes them Hidden Markov that a distribution has a common parameter that expresses that distribution with distributions that exist in each dimension of other multidimensional continuous distributions, and the commonalization relation of the parameters is individually specified in each dimension. A pattern recognition method comprising a model. 2. The pattern recognition method according to claim 1, wherein the multidimensional continuous distribution is a multidimensional normal distribution, and the common parameter is an average value and a variance value. 3. The pattern recognition method according to claim 1, further comprising a hidden Markov model in which the number of common parameters differs depending on the dimension. 4. When modifying the hidden Markov model by learning or the like, modifying common parameters in some models and interlocking modifying the common parameters included in all models. The pattern recognition method according to claim 1 or 2, wherein.