JPH08123465A - Adapting method for acoustic model - Google Patents

Abstract

PURPOSE: To increase a recognition rate with a small learning speech and a small calculation quantity. CONSTITUTION: A semicontinuous distribution HMM model is generated by using a learning speech for an unspecified speaker, its base distribution is stored in a code book 15, and weight coefficients for respective base distributions are stored in a weight coefficient memory 16; and weight coefficients as to all phonemes independent of respective phonemes are stored as weight coefficients 19 for all phoneme models. The learning speech of a recognition speech is inputted, the weight coefficients 19 for all the phoneme models are used, and only the respective base distributions in the code book 15 are adapted and stored in a code book 17. At the time of recognition, an input sound is recognized by using the weight coefficients in the code book 17 and the weight coefficients in the weight coefficient memory 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used as a standard pattern in speech recognition, and an acoustic model learned using speech (learning speech) recorded in another environment in advance is used as a specific speech. The present invention relates to a method of adapting to a voice collecting system line characteristic and a voice having a different characteristic from a learning voice such as a specific speaker.

[0002]

2. Description of the Related Art Hidden Markov Model (Hidede), which is a method for modeling acoustic characteristics of speech stochastically and statistically.
n Markov Model: HMM), a speech recognition system using one recognition target category, that is, a phoneme,
One or a plurality of HMMs are set for each vocabulary (or recognition target unit) such as syllables and words, and learning is performed using the learning voice, that is, an HMM is created. Upon recognition,
The probabilities that the input speech of the speech recognition system is observed from these models are calculated, and the recognition result candidates are arranged in the order of highest likelihood. Since the HMM is a statistical model, the strength of associating a certain acoustic feature quantity with a certain category is internally expressed as a probability distribution according to the frequency of appearance in the learning voice. That is, as shown in FIG. 4A, for all recognition target categories (for example, phonemes),
An initial state (around the beginning of the phoneme) a, a second state b, a third state c, and a final state (the end of the phoneme) d are sequentially transitioned, and each state is the acoustic feature amount of that phoneme in that state. Acoustic models M _{1 to MM} that represent a statistical distribution and are given transition probabilities from state to state are calculated in advance, and the probability that the input speech is output from a certain acoustic model is calculated, and the acoustic corresponding to the input speech is calculated. Find the likelihood of the model.

The HMM is based on the method of expressing the probability distribution, and
Cloth model, continuous distribution model, semi-continuous distribution model
Largely classified. In the discrete probability distribution model, the sound of speech
Resonance features are represented by coded discrete values.
For example, as shown in FIG.
Typical N feature vectors A₁~ A_NRepresented by either
And these feature vectors A₁~ A_NEach has a code
(Eg number) C₁~ C _NIs given. Also each sound
Acoustic model M showing elementary₁~ M_MFigure 5 for each of
As shown in B, code C₁~ C_NFor each of
Output probability P₁~ P _PThere is a one-to-one correspondence. Entering
The feature vector of the force voice is
Cutle A₁~ A_NWhich of the following is closest to the
The input speech is converted into a code string that represents the representative feature vector.
And the code string is for each acoustic model M₁~ M_MEach of
, The output probability is calculated. These are calculated
With the highest (most likely) acoustic model among the output probabilities
The corresponding phoneme is output as the recognition result.

In the continuous probability distribution model, acoustic feature vectors are used.
Cuttles are treated as continuous quantities. For example, the acoustic model in FIG. 4A
Le M_MThe initial state a is the acoustic feature distribution D₁
And the acoustic features of the states b to d, respectively.
Is the distribution D₂~ D_FourIs represented as Continuous probability distribution model
There are a single distribution model and a mixed distribution model.
Model A of 4A_MIs a mixture distribution model, for example, in FIG.
As shown in B, one mixture distribution D₀Has multiple distributions V₁
~ V₃It is expressed in the form of weighted addition of. These distributions V
₁~ V₃Is a Gaussian distribution that approximates the distribution of audio acoustic features.
Similar, average μ ₁~ Μ₃And the covariance matrix σ₁~ Σ₃Stop
Each is expressed. Acoustic model M₁~ M_MRespectively
As shown in FIG. 5C, a plurality of distributions for each state,
Although not shown in the figure, the weighting factor and are given and expressed.

The input speech is represented by each acoustic model.
The output probability is calculated from the distribution of each state
Output probability, that is, the likelihood of each
The Dell phoneme is used as the recognition result. The mixture distribution model is precise
It is possible to estimate the distribution, but the number of parameters
Because of the large number, many learning sounds are required. Half
The continuous probability distribution model is a discrete distribution model and a continuous distribution model.
It is a model that has the features of the mixed distribution type of
You. That is, in the continuous distribution model of Gaussian mixture distribution,
Set the number of mixture distributions large enough, for example 256, and
The same distribution V for each acoustic model₁~ V_NAnd each sound
The weighting coefficient is used to distinguish between the sound models. For example, in FIG.
As shown in FIG.₁~ M_M
About the distribution V₁~ V_NThe weight W for each of
Each is given. Similarly for states b, c and d
However, each acoustic model M₁~ M_MFor each of
V ₁~ V_NIs given a weight W. Tsuma
Base distribution V₁~ V_NIs all acoustic models, all states
Weighting coefficient W for each state of each acoustic model_i
The value of is determined as a value unique to each phoneme. Input voice
Calculates the output probability for each acoustic model, and
The phoneme of the acoustic model is used as the recognition result. Semi-continuous models are separated
Feature vector A in the dispersion model₁~ A_NBase instead of
Distribution V₁~ V _NWas used for the single unit shown in FIG. 5A.
Parameter space is represented by one codebook
Due to the characteristics of the discrete model of
A mixture distribution type continuous model in which the phoneme model is represented in detail.
It also matches the characteristics of Dell.

In voice recognition using a statistical model such as HMM, it is premised that the learning voice for estimating the model parameters and the voice to be actually recognized are picked up under similar conditions. I am trying. That is, it is assumed that the acoustic environment, for example, the background noise and the line characteristics are almost the same during learning and recognition. When the sound collecting conditions at the time of learning and at the time of recognition are different, there is a problem that the recognition accuracy is deteriorated because the acoustic feature amount of the voice actually to be recognized is different from the acoustic feature amount expressed by the model.

Variations in the acoustic feature quantity during learning and during recognition include additive effects on the spectrum and filter effects. Since background noise is added to the voice as power, it becomes additive even in the spectral domain. On the other hand, the difference (distortion) in the line characteristics changes the shape of the spectrum envelope, and usually changes the slope of the spectrum envelope, so that it has a filter effect in the spectrum region.

In the case where the acoustic conditions at the time of learning differ from the acoustic conditions at the time of recognition, attempts have been made to improve the recognition performance by adapting the recognition system to the acoustic conditions to be recognized. The two methods proposed so far will be described below. The first is a method called the cepstrum mean value normalization method. The cepstrum defined by the inverse Fourier transform of the logarithmic spectrum is often used as the acoustic feature of the voice. In the cepstrum domain, the filter in the spectrum domain is realized by addition and subtraction, so that distortion due to fluctuations in the line characteristics can be corrected by addition and subtraction of the cepstrum. A simple and effective line characteristic correction method based on this principle is the cepstrum average value normalization method. When the cepstrum coefficient is used as the acoustic feature amount of the voice, the time-invariant frequency spectrum tendency can be flattened by subtracting the average value over the voice section from the time series of the cepstrum. However, in the cepstrum average value normalization method, the principle is to flatten by subtracting the time-invariant spectrum envelope in the line by long-term averaging.
Also, since the average value of the time series of the cepstrum in a certain section is simply subtracted, there is a limit to the improvement effect such as an estimation error caused by the magnitude of the voice energy or the difference in the SN ratio. Met.

The second is a model adaptation method by conversion of a codebook. This method has been proposed for speaker adaptation, but if it is based on a model that uses a codebook, it is generally considered to be applicable as an adaptation method for the discrepancy between the recording environment of the learning speech and the recognition target speech. With this method, in the case of a discrete probability distribution model or a semi-continuous distribution model, the model can be adapted by converting the codebook obtained from the training speech from that obtained from the speech to be recognized. Is. This method will be described by taking as an example the case where a model learned by the voice of the line A which is the recording line of the learning voice is adapted to the voice of the line B which is the recording line of the recognition target voice. When there is a voice of the line A and a voice of the line B, the voice of the line A is used for the codebook A, and the voice of the line B is used for the codebook B.
Design each. Then, the voice of the line A is vector-quantized using the codebook A, and the HMM is learned using the resulting codebook A code sequence (HM
Create M). Next, the voices of the line B having the same utterance content are vector-quantized using the codebook A and the codebook B, respectively, and the correspondence between the codes of the codebook A and the codebook B is obtained by DP matching. When the voice of the line B is to be recognized, vector quantization is performed by the codebook B, the result is converted from the correspondence between the codebook A and the codebook B into the code sequence of the codebook A, and the codebook A is used. It becomes possible to recognize the voice of the line B by using the HMM learned by learning. However, this method requires that the voice of the line B, that is, the voice of the line in which the voice to be recognized is recorded, be large enough to design a codebook, and the same utterance as that of the line A is produced. The problem is that there must be audio for the content. Therefore, there is a need for an adaptation method with a smaller amount of adaptation speech and less restrictions on the utterance content.

[0010]

SUMMARY OF THE INVENTION An object of the present invention is to perform adaptation so as to obtain high recognition accuracy even when the learning voice and the voice to be recognized have different properties by using a small amount of learning voice. Another object of the present invention is to provide a method of adapting an acoustic model that can be performed with a small amount of calculation.

[0011]

According to the present invention, an acoustic model is constructed by a codebook in which a parameter space is represented by a plurality of basis distributions, and a weighting coefficient for each basis distribution in the codebook. Adapting the basis distribution expressing the parameter space by re-estimating the voices with different characteristics, that is, the voices at the time of recognition, using the all-category acoustic model learned independently of the recognition target category. .

According to the second aspect of the present invention, the change of each re-estimated basis distribution with respect to the pre-estimated basis distribution is used as an adaptation vector, each basis distribution is clustered according to the speech power, and the adaptation vector is defined as above. The base distributions belonging to each cluster are averaged, and each base distribution of the cluster is adapted using the averaged adaptation vector.

According to the third aspect of the present invention, the averaged adaptation vector for each cluster and the adaptation vector for each basis distribution of the cluster are weighted averaged, and the basis of the cluster is calculated using the weighted average adaptation vector. Adapt the distribution.

[0014]

With the above configuration, (1) by using the whole phoneme HMM, it is possible to adapt by any utterance irrespective of the utterance content of the adapted voice, and (2) considering the power of the voice. Therefore, there is an advantage that the line characteristics can be more accurately adapted. That is, when the power of the voice is high and the SN ratio is high, the effect of additional noise is small, and when the power is low, the opposite is used. The codebook is adapted so that the average value of the correction amount of the base distribution (codeword) belonging to the cluster is emphasized, and the correction amount of the base distribution itself is emphasized for the base distribution (codeword) belonging to the cluster with low power. Is possible.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS As an embodiment of the present invention, a case in which an acoustic model learned by a voice recorded in an environment with relatively good acoustic conditions such as a soundproof room is adapted to a telephone voice having a characteristic different from the learned voice A description will be given with reference to the drawings. In this example, a case where a semi-continuous HMM is used as an acoustic model will be described. The method of the present invention can be applied to both a discrete distribution HMM and a continuous distribution HMM as long as the model parameter space is expressed by a set of base distributions and the base distributions are shared by the models.

FIG. 1 shows a voice recognition device to which the present invention is applied. The analog voice signal from the input terminal 11 is converted into a digital voice signal by the voice input unit 12, and the acoustic feature amount (eg, cepstrum, Δ) is converted from the digital voice signal.
Cepstrum, Δ power, etc.) is the acoustic feature quantity extraction unit 13
It is extracted with. When the HMM is used as the acoustic model, the parameters of the HMM (the average value of the acoustic feature amount vector, the covariance, the transition probability) and the weighting coefficient of each distribution are calculated by the calculation unit 14. The semi-continuous distribution HMM is composed of a codebook in which the parameter space is expressed by a plurality of basis distributions and a weighting coefficient for each basis distribution in the codebook. The HMM learned by speech recorded in the relatively good environment. Are stored in the unspecified codebook 14, and the weighting coefficient for each base distribution for each HMM is stored in the weighting coefficient memory 16. Further, in the present invention, the base distribution obtained by adapting the base distribution of the codebook for unspecified speakers with the telephone voice is stored in the adaptation codebook 17. The recognition result is output from the calculation unit 14 to the output terminal 18. The acoustic feature quantity extraction unit 13 may be realized by hardware or software. When it is realized by software, it may be realized by the arithmetic unit 14 as long as the arithmetic unit 14 has sufficient arithmetic capacity.

The set of base distributions before adaptation, that is, the set of base distributions stored in the unspecified speaker codebook 15 is composed of V _{1 to} V _N as shown in FIG. 2A, for example. As described above, the semi-continuous HMM has a weighting coefficient for each distribution of this codebook, and the likelihood for the input speech is obtained by weighted addition of the probability distribution function values of each distribution. The size of the codebook 15, that is, the number of base distributions is often about 256 when a cepstrum coefficient is used as the acoustic feature amount.
The feature vector of the input speech is x, the probability density function value of each basis distribution is V ₁ (x), V ₂ (x), V ₃ (x), ..., _VN
(X) and the weighting coefficient for each distribution is
Assuming W ₁ , W ₂ , W ₃ , ..., W _N , the likelihood F (x) for the feature vector x of the input voice is F (x) = W ₁ V ₁ (x) + W ₂ V ₂ (x) + W ₃ V ₃ (x) + ... + W _{N VN} (x) (1) W _{1 to} W _N are different values for each HMM.

Params that determine the shape of the HMM corresponding to each phoneme
Meter (V₁, V₂, V₃,,, V _NThe Gaussian distribution of
Mean and covariance, weighting factor for each distribution,
W₁, W ₂, W₃,…, W_N) Is a lot of voice data
Using forward-backward algorithm
Is determined. Where the basis distribution V₁~ V_NIs all models, all
Shared across states, for each state in each model
To W_iThe value of is estimated as a value unique to each phoneme model.

In the adaptation according to the present invention, W of each model is
_i, That is, leave the contents of the weighting coefficient memory 16 unchanged
Every, base distribution V₁~ V_NOnly adapt. For adaptation
Better represent the parameter space of the speech to be recognized.
Each base distribution V ₁~ V_NThe mean and variance of
Change. The position moves due to changes in the mean value, and the covariance
The size of the distribution changes due to the change. With this adaptation
Distribution V₁~ V_NIs varied as shown in FIG. 2B.
If you do not get enough adaptation speech, the covariance is
You may change only an average value, without changing. Base distribution
V₁~ V_NMoving itself to a new parameter space
Model-specific W_iPhoneme model even if the
Is adapted to the new parameter space.
You.

A specific method of re-estimating the mean value and covariance by adaptation will be described assuming that the model is set in a phoneme unit. Re-estimation of the mean value and covariance is for the whole phoneme H
Performed using MM. In other words, independently of each recognized phoneme,
A weighting coefficient W ₁ for a model learned by using all speeches to be recognized and learned so as to have a relatively large likelihood for all phonemes, that is, a so-called whole phoneme model.
Re-learn only codebook 15 (mean and covariance) using ~ W _N. Generally, each phoneme model has a high weighting factor for 256 codewords, that is, a plurality of base distributions that are important for expressing the phoneme, and is close to 0 for others. Indicates a very small weighting factor value. Therefore, in re-estimation of individual phoneme models, codewords with large weighting factors (base distributions) move more, and codewords with smaller weighting factors (base distributions) move less, thus balancing the entire codebook. Since re-estimation cannot be performed well, re-estimation is performed using all-phoneme HMMs that have well-balanced weighting factors for all codewords (all basis distributions). This all-phoneme HMM is learned in advance when learning the codebook 15 for unspecified speakers and the weighting coefficient, and the weighting coefficient is stored in the memory 16 as the weighting coefficient 19 for all-phoneme model and adapted. Codebook 17
When this is created, the weighting coefficient for the whole phoneme model is used, and the others are adapted by estimating the average value and covariance of each phoneme model (HMM) by the forward / backward algorithm as in the case of normal learning. Create the codebook 17.

Since the all-phoneme model is independent of phonemes, it can be learned without depending on the utterance content. Therefore, it is also advantageous that there is no need for a constraint condition that a certain learning voice for adaptation should be uttered. . The calculation of the adaptive learning described above is performed in the calculation unit 14 in FIG. The adaptive learning speech on the line B does not need to be labeled with a section corresponding to each recognition category (phoneme).
The adaptive learning speech of is converted from an analog speech signal into a digital speech signal in the speech input unit 12 and the acoustic feature quantity extraction unit 13 to be an acoustic feature quantity vector. By using the acoustic feature amount vector of the speech section of the adaptive learning speech of the line B as an observation sample, it is possible to re-estimate the average value, covariance, and weighting coefficient of the distribution of all phoneme HMMs by the forward / backward algorithm. Each phoneme HMM
Does not need to re-estimate / update the weighting factors, only the codebook 15 needs to be adapted. In this way, the weighting coefficient of the basis distribution is the same as the original model for unspecified speakers, that is, the content of the memory 16, and the average value and covariance of the codebook 15 are optimized for the voice of the line B. Is created and used as an adaptation codebook 17.

In general speech recognition by an unspecified speaker, a model for the unspecified speaker in which the acoustic feature amount is represented by the codebook 15 for the unspecified speaker and the weighting coefficient memory 16 is used. When recognizing the voice from the line B, the HMM adapted to the line B to be recognized by the adaptive codebook 17 and the weighting coefficient memory 16 is used, and the likelihood of the HMM of each recognition category with respect to the input voice of the line B is calculated. The model category with the highest likelihood is used as the recognition result, or the recognition result candidates are selected in the descending order of likelihood.

FIG. 3A shows a microphone sound according to the method of the present invention.
Phone voice of semi-continuous HMM codebook 15 learned in
Phoneme recognition results for the telephone speech when it is adapted to
Is shown. Acoustic features are cepstrum and delta cepstrum
12 dimensions each. In the figure, CMN is described in the section of conventional technology.
Cepstrum mean normalization method, mean is each basis distribution
Of mean and v
ar. Is an adaptation of mean and covariance at the same time, me
an + var is the covariance after adapting only the mean
This is an adapted case. From this figure to CMN
When combined with the bright method, the recognition rate improved to 55.4%.
Was.Embodiment of the invention of claim 2 The semi-continuous HMM codebook 15 is used for the speech pattern of each basis distribution.
Clustering according to the word. That is, the voice
The base distributions that are close to the word belong to the same cluster. Claim
Each basis distribution (code) obtained in the embodiment of the invention of Item 1
Adaptive basis distribution (codebook) corresponding to Book 15)
When the change in 17) is used as an adaptation vector,
Average the adaptation vector for each cluster
The representative adaptation vector of the cluster (average adaptation vector
, And all the basis distributions belonging to that cluster.
It is adapted by the representative adaptation vector of the cluster. An example
For example, by voice power clustering, for example, in FIG. 2A.
Base distribution of V₂, V₃, V ₆Belong to the same cluster
Then, the basis distribution V₂, V₃, V₆Adaptation code of
Change vector (adaptive
Vector) E₂, E₃, E₆(In this case, change the mean
Averaging vector)
Toru E_m, The basis distribution belonging to that cluster
V₂, V₃, V₆To adapt. In this case a kind of smoothing
Due to the effect of optimization, robust adaptation is possible even when the amount of adaptation voice is small.
Adaptation can be expected.

FIG. 3B shows the recognition result when the codebook is adapted by the method of the present invention. In addition to the cepstrum and the Δ cepstrum, the normalized logarithmic power and its first derivative (Δ power) were used as acoustic features. The clustering was performed by the normalized logarithmic power. The optimum number of clusters was experimentally determined and set to 5. The recognition rate is improved overall compared to the previous experiment due to the increased feature quantity, but when clustering by power, CMN and m
The recognition rate is higher than that of ean (when the average value of the codebook is adapted by the whole phoneme HMM). Embodiment of the Invention of Claim 3 A weighted linear sum of the adaptation vector corresponding to each basis distribution and the representative adaptation vector (averaged adaptation vector) of the cluster to which the basis distribution belongs is newly added. Adapt the codebook as an adaptation vector.
The effect of filter distortion is considered to be the main cause of deterioration of accuracy in the case of high voice power, and the effect of additive noise cannot be ignored in the case of low voice power. It can be expected that more accurate adaptation can be realized by manipulating the contribution rate of the adaptation vector corresponding to itself and the representative adaptation vector of the cluster.

Although the present invention is applied to the case of adapting to the line voice in the above, it can be applied to so-called speaker adaptation. The acoustic model is not limited to HMM.

[0026]

As described above, according to the present invention,
(1) It is possible to adapt the acoustic model to the characteristics of the speech to be recognized by the adapted speech of arbitrary utterance content,
(2) More robust and highly accurate adaptation is possible by performing adaptation according to speech power. (3) Re-estimation of the common codebook only, without re-estimating the distribution coefficient of each category model. Therefore, less learning speech is required for adaptive learning, and therefore, there is an advantage that the calculation time is shorter.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a voice recognition system to which the invention is applied.

FIG. 2 is a diagram showing a state of adaptation of an acoustic model according to the present invention.

FIG. 3 is a diagram showing the effect of the present invention.

FIG. 4A is a diagram showing an example of an acoustic model. B is a diagram showing an example of a mixture distribution.

5A is a diagram showing an example of a codebook of a discrete distribution model, B is a diagram showing an example of each acoustic model thereof, and C is a diagram showing an example of a continuous distribution model.

FIG. 6 is a diagram showing an example of a semi-continuous distribution model.

Claims (3)

[Claims] 1. A learning voice is used to extract an acoustic feature amount of the voice, and the feature amount is statistically modeled,
In a method of adapting an acoustic model corresponding to a recognition category by using a voice having a property different from that of the learning voice at the time of recognition, the acoustic model is a codebook expressing a parameter space with a plurality of base distributions. , The weight distribution for each basis distribution in the codebook, and using all-category acoustic models learned independently for each recognition target category, the basis distribution expressing the above parameter space is made different in the above properties. An acoustic model adaptation method characterized by re-estimation and adaptation by speech. 2. A change of each re-estimated basis distribution with respect to a pre-estimated basis distribution is set as an adaptation vector, the basis distribution is clustered according to speech power, and the adaptation vector belongs to each of the clusters. The acoustic model adaptation method according to claim 1, wherein the basis distributions are averaged and each of the basis distributions of the cluster is adapted using the averaged adaptation vector. 3. The change of each re-estimated basis distribution with respect to the pre-estimated basis distribution is used as an adaptation vector, the basis distribution is clustered according to speech power, and the adaptation vector belongs to each of the clusters. The basis distribution is averaged, the averaged adaptation vector for each cluster and the adaptation vector for each basis distribution of the cluster are weighted averaged, and the weighted average adaptation vector is used to calculate the basis distribution of the cluster. The method for adapting an acoustic model according to claim 1, characterized in that