JPH11143486A - Device and method adaptable for speaker - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize the speaker-adaptable capacity excellent in accuracy in the speaker adaptation using the maximum posterior probability estimating method. SOLUTION: A set 17 of a large number of speaker models independent from an adaptable speaker is prepared using an acoustic analyzing means equivalent to an acoustic analysis part 10. Then, the sound of the adaptable speaker is inputted, and analyzed by the acoustic analysis part 10, the distribution of the feature parameter vector of the adaptable speaker is obtained, and preserved as the sample data 16 for adaptation. An datable model preparation part 15 measures the distance between the adaptable speaker model preserved as the sample data 16 for adaptation and a large number (N-pieces) of speaker models preserved as the set 17 of the speaker model, and the speaker models of M pieces are selected in the order of smaller distance to the adaptable speaker model. The weighted addition of the selected speaker models of M pieces is achieved to determine the initial model.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speaker adaptation technique for correcting an initial speech model using a speech uttered by a speaker to be adapted as learning data and creating a speech model adapted to the speaker. .

[0002]

2. Description of the Related Art A hidden Markov model (Hidden Ma)
rkov Model (hereinafter abbreviated as HMM) is widely used in speech recognition because it can easily deal with spectral and temporal fluctuations of speech and can realize high recognition accuracy. The HMM is a state transition model having transition probabilities between states and output probabilities of symbols associated with state transitions. To model a signal that changes continuously with time, such as a speech signal, it is necessary to model from left to right. A so-called left-to-right type model in which the state transitions is appropriate.
FIG. 1 shows an example of a left-to-right type HMM in the case of four states. Here, 1, 2, 3, and 4 represent states, and a _ij (i, j = 1,..., 4) represents states i to j
Indicates the probability of transition to. B _j (o) indicates the probability that the symbol o is observed in the state j with the state transition. In the case of speech recognition, a feature parameter vector is usually used as this symbol.

In speech recognition using an HMM, a unit of speech to be recognized (eg, phonemes, phonemes, syllables, words)
In each case, a speech model using an HMM is prepared, and a model such as a state transition probability a _ij and a feature parameter vector output probability b _j (o) is prepared using speech data for training in advance.
Determine parameters. At the time of recognition, the input speech is analyzed, converted into a sequence of feature parameter vectors, and a sequence of speech units corresponding to a model that is most likely to observe the sequence of parameter vectors is determined. The result is the recognition result.

In a statistical model such as an HMM, generally,
By increasing the data for learning, the accuracy of the model can be improved. Therefore, by using a large amount of data uttered by the target speaker, a highly accurate model for the speaker can be created. However, for this purpose, it is necessary to collect a large amount of voice data before use, which requires a large amount of utterance from the speaker, which has been a practical obstacle.

On the other hand, it is conceivable to collect speech data of a large number of unspecified speakers in advance and create a standard speaker model independent of the speakers based on the collected speech data. The use of the model for unspecified speakers has the advantage that the user does not need to make a large amount of utterances to train the speaker model prior to use, and can immediately start using the model. However, since the model is not adapted for each speaker, recognition accuracy is not sufficient. In general, in speech recognition, the accuracy of recognition can be improved by preparing a speech model that suits the individual, but in order to create that speech model, it is necessary to prepare a large number of utterances in advance. Accuracy and labor are in a trade-off relationship.

[0006] From the above, it is possible to start using only a small amount of speaker-specific utterance data, and to improve recognition accuracy sequentially by adding speaker-specific utterance data. "Speaker adaptation" has attracted attention in some aspects of applying speech recognition. For something that is used personally to some extent, such as dictation, it can be used with a simple procedure, and it is desirable that the accuracy of recognition be improved as it is used. Will be valid.

As one of the methods of "speaker adaptation", there is a method of realizing an initial speech model obtained in advance by taking in the characteristics of an actual speaker and a use environment and modifying it. Above all, attempts have been made to re-estimate the parameters of the speech model based on Bayesian estimation, which has been effective. Bayesian estimation determines parameters based on Bayes' theorem, and is based on the following concept.

The cause H that caused the event A
_i possibility P (H _i | A) is the Bayes' theorem, the cause of the probability P (H _i), events from the cause probability generated P | calculated by multiplied by (A H _i) Is done.
At this time, P (H _i ) is called prior probability, and P (H _i | A)
Is called the posterior probability. That is, it is assumed that the posterior probability is determined by the prior probability P (H _i ) predicted in advance and the sample probability P (A | H _i ).

The distribution parameters are taken as the cause, and the resulting sampled data is taken. Then, the posterior probabilities of the parameters of the distribution are obtained from the prior probabilities that are the predicted values of the parameters of the distribution and the probabilities of the results obtained from the sample data. Thus, if there is a correct prediction for the parameters of the model, it can be effectively incorporated into the determination of the posterior probability.

In general, if the prior probability and the posterior probability belong to the same distribution family with respect to the probability of the result, the sample only causes a transformation within the distribution family, which facilitates mathematical treatment. The distribution family of the prior and posterior probabilities at this time is called a natural conjugate distribution, but has a Gaussian distribution N (θ,
σ ² ) mean Gaussian distribution N
(Λ, τ ² ) is known to be a natural conjugate distribution.

In a framework based on Bayes' theorem, a method of estimating parameters and performing speaker adaptation is a method of estimating a maximum posterior probability, for example, Chin-Hui Lee, Ch
ih-Heng Lin and Biing-Hwa
ng Jung, "A Study on Spaa
ker Adaptation of the Par
meters of Continous Den
site Hidden Markov Model
s ", IEEE Transactions on S
signal Processing, Vol. 39, N
o. 4, April, 1991 (hereinafter referred to as Reference 1)
And is disclosed. According to this, if the average parameter μ follows the prior probability P _o (μ) and its variance σ ² is known and fixed, the conjugate prior distribution of μ has an average value ν and a variance τ
It is a Gaussian distribution with ² , using which the maximum posterior probability estimate of the parameter is

[0012]

(Equation 1) Given by Here, n is the number of training samples observed in the corresponding HMM state, and o is the average of the samples. That is, the estimated value of the average value by the maximum posterior probability estimation method is given by a weighted average of the average value ν of the prior distribution and the average o of the samples. As is apparent from (1), when n is 0, that is, when there are no samples, the estimated value remains the mean value ν of the prior distribution. When n is sufficiently large, that is, when the number of samples is sufficiently large, the estimated value approaches the average o of the samples. In this way, the maximum a posteriori probability estimation method takes in the distribution predicted in advance and can asymptotically approximate the model parameters to the characteristics of the training sample according to the number of sample data. The accuracy of the successive speaker model can be improved, and ideal speaker adaptation can be realized.

[0013] Document 1 describes a case of variance and a case of both average and variance as examples of speaker adaptation to other parameters based on Bayesian estimation. It is shown to be highly effective.

[0014]

A speaker adaptation method using parameter estimation by the maximum posterior probability estimation method has been disclosed. In the prior art 1 disclosed in Japanese Patent Application Laid-Open No. 8-95592, speaker adaptation for matching a phoneme model to a specific speaker using a standard phoneme model is performed. As the model, an unspecified speaker model that has been trained in advance using voice data uttered by various speakers of various ages, and that recognizes the voices of an unspecified number of speakers is used. In the adaptation method using the unspecified speaker model shown in the conventional example 1, the distribution of the standard initial speech model used as the prior distribution is widened, and it is sufficiently compensated with a small amount of training data. It is difficult to correct parameters to adapt to the speaker to be adapted.

Further, in the conventional example 2 disclosed in JP-A-8-110792, an initial speaker model is created using a speaker cluster created by a tree structure clustering model. With this method, the distribution of the initial speaker model can be narrowed down to some extent. However, if the center of the cluster deviates greatly from the vector of the speaker model to be adapted, the effect of the correction is small, and May not be able to adapt.

The maximum posterior probability estimation method is, as described above,
Using the speaker's utterance sample data, the parameters of the initial speaker model are modified and adapted to the speaker, and the accuracy of the model can be gradually improved according to the number of sample data. The choice of the early speaker model has a significant effect on the ability to adapt. The present invention has been made in view of such a point, and provides means for realizing a speaker adaptation capability with high accuracy in speaker adaptation using a maximum posterior probability estimation method. The present invention provides a speaker adaptation apparatus that creates an accurate speech model for a particular speaker by selecting an optimal initial speaker model using a small amount of utterance data from a specific speaker to be adapted. It is an object of the present invention to realize a speech recognition device having a speaker adaptation capability with high accuracy by using the speaker adaptation device.

[0017]

According to the present invention, there is provided an apparatus for re-estimating parameters of a speaker model by a maximum posterior probability estimation method using an initial speaker model and data for adaptive learning, and performing speaker adaptation. Many (N) created from many speakers
Speaker models are prepared in advance, and among these many speaker models, M speaker models which are close in distance to the speaker to be adapted are selected, and the selected M It is characterized in that an initial speaker model is formed by mixing individual speaker models. Further, at this time, the number M (M << N) is determined based on the distance relationship between the adaptation target speakers and N individual speaker models prepared in advance. This makes it possible to determine an initial model with high accuracy according to the distribution, and to suppress an unnecessary increase in parameters.

In this configuration, the N speaker models to be mixed may be weighted according to the distance from the feature vector of the speaker to be adapted.

When the distance between the speaker to be adapted and the speaker model closest to the distance is set as the base distance, the distance between the speaker to be adapted and the base distance is within a certain range as compared with the base distance. By selecting a speaker model, the number N of speaker models to be mixed may be made variable.

The present invention can also be realized as a method, and at least a part thereof can be realized as a computer program product.

[0021]

Embodiments of the present invention will be described below.

FIG. 2 is a block diagram showing an embodiment of a speaker adaptation apparatus and a speech recognition system using the same according to the present invention. In the speech recognition system, the input speech is analyzed by the acoustic analysis unit 10, and a sequence of feature parameter vectors is extracted. The extracted feature parameter / vector sequence is compared with the speech model 13 in the phoneme matching unit 11 while referring to coarse information from the language model 14.
Create multiple phoneme sequence candidates. The plurality of phoneme sequence candidates thus generated are input to the language recognition unit 12 by the language model 14.
Re-evaluate using the detailed information from to determine the final recognition result. The means for generating the above-mentioned speaker-adapted speech model 13 is the speaker adaptation apparatus according to the present invention, and the method and configuration thereof will be described below.

In this embodiment, first, the voices of a large number of speakers are collected in advance, and a large number of voices independent of the adaptation target speaker are collected using acoustic analysis means (not shown) equivalent to the acoustic analysis unit 10. A set 17 of speaker models is prepared. That is, N
By collecting voices of human speakers, N distributions are calculated and prepared in advance. FIG. 3 schematically shows output probability distributions of feature parameter vectors of these many speaker models by broken lines. For convenience of explanation, the feature parameter vector is displayed as two-dimensional. However, this feature parameter vector is usually a multidimensional vector of about 30 dimensions. As shown in FIG. 3, the speaker models are distributed in a multidimensional space while overlapping each other. Next, for speaker adaptation, the speech of the speaker to be adapted is input and analyzed by the acoustic analysis unit 10 to obtain the distribution of the characteristic parameter vector of the speaker to be adapted, and stored as the sample data 16 for adaptation. I do. FIG. 4 is a solid line showing an example of the distribution of the output probabilities of the feature parameter vectors of the adaptation target speaker obtained from the adaptation sample data. It shows the relationship with the distribution. As described above, the speaker model to be adapted and a large number (N) of speaker models are determined, and the adaptive model creation unit 15 creates the adapted speech model 13. It is done in the following steps.
The adaptation model creating unit 15 first stores a large number (N) of speaker models to be adapted stored as adaptation sample data 16 and a set 17 of speaker models.
Measure the distance between the speaker model. Assuming a multivariate Gaussian distribution as the speaker model, the Mahalanobis distance between the center vector of the speaker model to be adapted and the specific speaker model can be used as this distance. Using this distance, M speaker models are selected in order of decreasing distance from the speaker model to be adapted. FIG. 4 illustrates how the nth, n + 1th, and n + 2th models, which are closest to the speaker model to be adapted, are selected when M is 3. In this way, when M speaker models can be selected, an initial model is determined as a weighted addition value.

The weight at this time can be determined based on these distances.

Further, the number M of the models to be mixed may not be a fixed value but may be changed according to the distribution. If you are unsure which model to select as an initial model candidate,
The number M of models to be mixed is increased, and if it is certain, the number M of models to be mixed is reduced. For this purpose, the Mahalanobis distance between the center vector of the speaker model to be adapted and the N specific speaker models is measured, and the distance is within a certain range compared to the shortest distance among all models. Choose a model. When the number of selected models is M, an initial model is created by mixing the M models by weighted addition in the same manner as in the previous example.
At this time, as in the previous example, the weight can be determined based on these distances. By doing this,
For ambiguous, uncertain initial model candidates,
A fine model can be created by increasing the number of mixtures, and conversely, with respect to initial model candidates with low ambiguity and high accuracy, it is possible to create a model in which the number of mixtures is reduced to reduce the amount of calculation. The within a range of above, when the shortest in distance to the center vector and all models of the adaptive target speaker model and D _min, those ratios of the distance and D _min is equal to or less than a predetermined value, or , The difference between the distance and D _min may be smaller than a certain value.

Hereinafter, specific examples will be described in detail.

[Specific Example 1] Here, a speaker adaptation method corresponding to continuous speech recognition will be described as an example. For continuous speech recognition, it is appropriate that the unit of speech as the unit of recognition be a phoneme level. Therefore, here, HMM
Create HMM is created using voice data
This is done by performing a procedure called training to determine the parameters of M. To train the HMM at the phoneme level using continuously uttered utterance data, the training is performed by connecting the final state of the HMM of the preceding phoneme to the initial state of the HMM of the succeeding phoneme for the uttered phoneme sequence. Do.

In order to implement the present invention, it is necessary to collect utterance data from a large number of speakers in advance and create an HMM for each speech unit that is a recognition unit. To model a signal that changes continuously with time, such as an audio signal, as described above, a so-called left-to-right type HMM in which the state transitions from left to right is appropriate. For example, a 4-state HMM such as 1 is used.
In FIG. 1, 1, 2, 3, and 4 represent states, and a
_ij (i, j = 1,..., 4) indicates the probability of transition from state i to state j. In state j, an event observed from the model is represented by b _j (o). Since the feature parameter vector of the speech that is the target of this observation event is a continuous signal, b _j (o) can be represented by the following continuous probability density function.

[0029]

(Equation 2) Here, o is an observation vector, and N [...] is a Gaussian distribution function of the mean vector μ _j and the variance / covariance matrix Σ _j .

Parameters in the individual models, a
_ij (i, j = 1,... 4) and b _j (o) (j = 1,.
Can be determined by iterative calculation using the known Baum-Welch method. This Baum-Welc
For re-estimation by the h method, see, for example,
ence Rabiner, Biing-Hwang
Juang, "Fundamentals of Sp.
tech Recognition ”, Prentic
e Hall PTR93. The outline is as follows. Baum-Welc
In the h method,

[0031]

(Equation 3) By re-estimating the parameters, replacing the estimated parameters and repeating the calculation of the estimation,
Can be created such that the probability of observing is maximized.

At this time, assuming a Gaussian distribution as in equation (2) as b _j (o), the re-estimation equation for μ _j and Σ _j is as follows:

[0033]

(Equation 4) Becomes Here, γ _t (j) is the probability of being in state j at time t given observation series o and model. As described above, a plurality of speaker models created in advance are determined.

The adaptation is performed in the following steps. First, speech data for adaptation learning is collected from adaptation target speakers. Then, the H of each phoneme included in the learning data for adaptation is calculated.
For MM, parameters are determined according to the method described above. At that time, with Viterbi segmentation,
Since the frame corresponding to each state of the phonetic HMM is determined, the number of samples of the adaptive learning data is obtained.

Next, the distance between the model of the speaker to be adapted obtained from the training data and a number of speaker models obtained in advance is calculated. The Mahalanobis distance between the average value vector and the speaker model is used as this distance.

The mean vector of the distribution of the j-th state of a certain k-th speaker model is μ _kj , and the variance / covariance matrix is Σ _kj
, And the average value vector μ _j of the j-th state of the speaker to be adapted and the Mahalanobis distance of this k-th model are D
_kj , the square of D _kj is

[0037]

(Equation 5) Is required. Here, Σ _kj ^-1 is the inverse matrix of the variance / covariance matrix Σ _kj , and (μ _kj −μ _j ) ^t is the vector (μ _kj −
μ _j ).

The Mahalanobis distance D _kj is calculated for all speaker models, and M speaker models are selected from the closest one. FIG. 4 shows that M is 3
In this case, three models, n, n + 1, and n + 2, are selected from the closest one. However, suffixes relating to the state are omitted.

Now, consider the feature vector of the mth (1 ≦ m ≦ M) th speaker model among the selected M speaker models. The average of a certain element of the feature vector is μ _mj , and the variance is τ _mj ² . Now, it is assumed that the average μ _j of the elements of the feature vector has a prior distribution P ₀ (μ), and the variance σ _j ² is a known fixed value. Then,
The conjugate prior to μ _j is a Gaussian distribution. Thus, if the m-th (1 ≦ m ≦ M) speaker model is a conjugate prior distribution with respect to the average, the maximum posterior probability estimation method
The average estimate μ _mj ^MAP is

[0040]

(Equation 6) Given by Here, n is the number of samples.

That is, using the m-th (1 ≦ m ≦ M) speaker model as an initial model, the estimated value of the average of an element having a feature vector is μ _mj ^MAP , the average value of a prior distribution μ _mj , and adaptive learning. the weighted average of the average o _m of the observed samples in.

Then, using the estimated values obtained for the m-th speaker model, the model for the speaker to be adapted is:
Given in the form:

[0043]

(Equation 7) Here, _cmj is the j-th state of the speaker model,
A weighting factor for the mth speaker, which can be determined based on the distance calculated in the above process. That is,

[0044]

(Equation 8) And it is sufficient. At this time, clearly

[0045]

(Equation 9) And (9) is determined to satisfy the statistical constraints.

[Specific Example 2] In the specific example 1, the number M of mixtures of the model is fixed. However, by changing the value of M according to the distribution of the model, the adaptability to the calculation cost is high, and the efficiency is high. Model can be created. That is,
The distribution of distances between the center vector of the speaker model to be adapted and a number of specific speaker models prepared in advance differs for each state, and when the distribution is not uniform, the value of M depends on the likelihood. It changes. For example, when the distance between the center vector of the speaker model to be adapted and a large number of speaker models prepared in advance is significantly different between the respective speaker models and is close to only a small number of speaker models,
It is sufficient to use only those models as a distribution for mixing, and by doing so, it is possible to prevent an increase in useless parameters.

In the specific example 2, the Mahalanobis distance D _k between the model of the speaker to be adapted and a large number of speaker models obtained in advance is determined, and the number of mixtures is determined based on the Mahalanobis distance D _k . In order to carry out the present invention, first, in the same manner as in Example 1,
Speech data from a large number of speakers is collected in advance, and HMMs for a large number of speakers are created for each voice unit as a recognition unit. Next, speech data for learning for adaptation is collected from the adaptation target speaker, and model parameters are determined for the HMM of each phoneme included in the training data for adaptation. Then, the distance between the model of the speaker to be adapted thus obtained and a number of speaker models is calculated. The Mahalanobis distance is used as the distance. In this specific example 2, the model of the speaker to be adapted is
The least distance between many speaker models

[0048]

(Equation 10) In this case, only the model k having a distance D _k within a specified range as compared with D _min is selected as an element to be mixed.

This situation will be described below with reference to the drawings. For example, as shown in FIG. 5, the distance to the model n D _n
Is the minimum, and if the distances D _{n + 1} and D _{n + 2} to the model n + 1 and the model n + 2 are within a certain range as compared with the minimum distance, 3 of the model n, the model n + 1 and the model n + 2
One model is chosen as the model to mix. Also,
For example, as shown in FIG. 6, the distance to the model n is the minimum, and the distance to the model n + 1 is within a certain range as compared with, but the distance to the next closest model n + 2 must be within a certain range as compared to. For example, only two models, model n and model n + 1, are selected as a mixed model.

As a method of determining a certain range in comparison with the minimum distance, the distance D _k to the model k and the minimum distance D _min
Is less than or equal to a certain value δ, that is,

[0051]

[Equation 11] One way is to select the k-th model.

The distance D _{k from} the model k and the minimum distance D
The difference from _min is less than or equal to the fixed value δ ', that is,

[0053]

(Equation 12) The k-th model can be selected.

When the number of selected models is M ′ in any of the methods, using the estimated values obtained for the selected speaker models, the model for the speaker to be adapted is: It is given in the following form as in the case of the specific example 1.

[0055]

(Equation 13) Here, _cmj is the j-th state of the speaker model,
The weighting factor for the second speaker, which can be determined based on the distances calculated in the above process. That is,

[0056]

[Equation 14] And it is sufficient.

[0057]

According to the present invention, there is provided an apparatus for re-estimating parameters of a speaker model by a maximum posterior probability estimation method using an initial speaker model assumed in advance and adaptive learning data, and performing speaker adaptation. As an initial speaker model, an attempt is made to assume a predicted distribution that is considered as close as possible to the characteristics of the speaker to be adapted. By using a prediction distribution as close as possible to the characteristics of the speaker to be adapted to the initial speaker model, highly accurate model parameter estimation is performed with a small amount of adaptive learning data, and good speaker adaptation is realized. In the present invention, the distance between a large number of specific speaker models obtained in advance and the speaker to be adapted is measured, and this function is effectively exhibited by selectively using N models close in distance. A method is disclosed by which high adaptability can be achieved.

Further, the present invention discloses a method of changing the number M of models to be selected according to the relative relationship of the distance to the models when selecting M models close in distance. This is to select the optimal mixture model according to the distribution of the assumed distribution to be assumed. This has the effect of exhibiting the best adaptability and reducing the amount of calculation processing.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating an example of an HMM.

FIG. 2 is a block diagram showing an embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of distribution of a number of specific speaker models.

FIG. 4 is a diagram illustrating an example of a distribution of adaptation target speakers obtained from learning data.

FIG. 5 is a diagram illustrating an example of a distribution of a specific speaker model located closest to an adaptation target speaker model;

FIG. 6 is a diagram illustrating another example of the distribution of a specific speaker model that is closest to an adaptation target speaker model;

[Explanation of symbols]

 1-4 HMM state 10 Acoustic analysis unit 11 Phoneme matching unit 12 Language recognition unit 13 Speech model 14 Language model 15 Adaptation model creation unit 16 Adaptation sample data 17 Set of specific speaker models 20 Feature vectors of many speakers 21 Distribution of feature vector of speaker to be adapted

Claims (4)

[Claims] 1. A speaker adaptation apparatus for re-estimating parameters of a speaker model by a maximum a posteriori probability estimation method using an initial speaker model and data for adaptive learning and performing speaker adaptation. A number of initial speaker models are created from the speakers, and the adaptation learning data uttered by the adaptation target speaker is extracted, and the features of the adaptation target speaker are extracted. The N speaker models are selected from the one closest to the feature of the speaker to be adapted in terms of distance, and each of the selected N speaker models is assumed to be a distribution assumed in advance, and the learning data for adaptation is used. A speaker adaptation apparatus comprising: estimating parameters of a speaker model using the parameters; and adding and adding N speaker models having the estimated parameters to generate a speech model of a speaker to be adapted. 2. The speaker adaptation apparatus according to claim 1, wherein the N speaker models to be mixed are weighted according to a distance from a feature vector of the adaptation target speaker. 3. When the distance between the speaker to be adapted and the speaker model closest in distance is the base distance, the distance between the speaker to be adapted and the base distance is within a certain range as compared with the base distance. 3. The speaker adaptation apparatus according to claim 1, wherein the number N of speaker models to be mixed is made variable by selecting a speaker model. 4. A speaker adaptation method in which parameters of a speaker model are re-estimated by a maximum posterior probability estimation method using an initial speaker model and data for adaptive learning to perform speaker adaptation. A number of initial speaker models are created from the speakers, and the adaptation learning data uttered by the adaptation target speaker is extracted, and the features of the adaptation target speaker are extracted. The N speaker models are selected from the one closest to the feature of the speaker to be adapted in terms of distance, and each of the selected N speaker models is assumed to be a distribution assumed in advance, and the learning data for adaptation is used. A speaker adaptation method comprising: estimating a parameter of a speaker model using the parameters; and adding and adding N speaker models having the estimated parameters to generate a speech model of a speaker to be adapted.