JP3256979B2 - A method for finding the likelihood of an acoustic model for input speech - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is used as a model in a speech recognition system, and extracts acoustic features of speech.
Input sound of an acoustic model whose features are statistically modeled
The present invention relates to a method for calculating likelihood for a voice .

[0002]

2. Description of the Related Art Hidden Markov method (Hidden method), which is a technique for stochastically and statistically modeling acoustic features of speech.
In a speech recognition system using a Markov Model (HMM), for each recognition target category, that is, for each vocabulary (or recognition target unit) such as a syllable, a word, and a phrase,
One or a plurality of HMMs are set, and learning is performed using a learning voice. At the time of recognition, the probability that the input speech of the speech recognition system is observed from those models is calculated, and the category of the model having the highest likelihood (likelihood) is determined as the category of the input speech, or in the order of the highest likelihood. It is a recognition category candidate. Since the HMM is a statistical model, the strength of associating a certain category with a certain acoustic feature is stored therein as a probability distribution according to the frequency of appearance in the learning speech.

On the other hand, in a speech recognition system, the speech to be recognized is often an utterance style (eg, fast, slow, loud, small, etc.) different from the learning speech, and sound collecting conditions (microphone, transmission characteristics, etc.). Is uttered in an ambient noise environment or the like, which causes a reduction in the recognition rate. In speaker-independent speech recognition, differences in speakers also reduce the recognition rate. H
MM can model very well the acoustic features that appear in the training speech, but can model well the acoustic features that appear infrequently or not at all in the training speech. Can not. Therefore, in the speech recognition system using the HMM, several methods are proposed to obtain a high recognition rate even for speech of various utterance styles (for example, fast, slow, large, small, question tone, etc.). It has been.

[0004] One of the methods is, in addition to ordinary utterances,
A method that intentionally collects utterances of various utterance styles as learning sounds, learns the HMM using the learning utterances, and attempts to construct a model robustly for various utterance styles. ^{(1) and (2)} . However, in reality, it is very difficult to collect voices of various utterance styles, and it is impossible to cover all different conditions such as utterance styles, sound pickup conditions, and ambient noise conditions. There is a limit to the recognition target speech that can be handled by this method. Another major problem is that the computation time required for HMM learning becomes enormous as the number of learning voices increases.

Another method is a method applicable only when the HMM is a discrete probability distribution model. In the case where the HMM is a discrete probability distribution model, a sequence of speech signals is encoded and represented. Therefore, a codebook for the encoding is converted by a method called codebook mapping ^{(3), and the} learned model is converted. Is a method ⁽²⁾ for adapting to other utterance styles. This method will be described by taking as an example a case where a model learned with a voice uttered at a normal speed is adapted to a voice uttered at a high speed. When there is a normal utterance rate and a fast utterance rate, a codebook 1 is designed using a normal utterance rate voice, and a codebook 2 is designed using a fast utterance rate. Then, the speech of the utterance at a normal speed is vector-quantized using the codebook 1, and
The resulting codeword 1 codebook sequence is HM
Learn with M. Next, the speech having the same utterance content and the normal utterance speed and the fast utterance speed are vector-quantized using the codebook 1 and the codebook 2, respectively.
And the corresponding relationship between each codeword of the codebook 2 and the corresponding codeword are obtained by DP matching. When a speech having a high utterance speed is to be recognized, vector quantization is performed in the codebook 2 and the result is converted into a codeword sequence of the codebook 1 from the correspondence between the codebook 1 and the codebook 2. 1 can be used to recognize a speech with a high utterance speed. However, this method is also based on the premise that voices of various utterance styles are provided as learning voices. That is, in the above example, both normal-speed utterances and high-speed utterances must be provided in such a large amount that a codebook can be designed. Codebook mapping cannot be performed unless voices with the same voice content but different voice speeds are obtained. Furthermore, if the codebook can be designed with a high-speed voice, it is possible to learn the HMM itself with a high-speed voice. Therefore, this method involves the problem of collecting learning speech as in the previous method. In addition, the HMM only needs to learn one utterance, but since the codebook needs to be designed for each utterance style, it takes a similar amount of calculation time as learning the HMM. Met. References (1) Lippmann, R .; , Et al. , "Mu
lti-style Training for ro
bus isolated-word speech
recognition, "Proceedings
of International Conference
nice on Acoustics, Speech a
nd Signal Processing '87,
17.4, pp. 705-708, 1987 (2) Miki, et al., "Speech Recognition Using Adaptation to Utterance Variation", IEICE Technical Report, SP90-
19, 1990.6 (3) Shikano, K .; , Et al. , "Spe
aker-Adaptation through V
vector Quantization, "Proce
edings of International C
onence on Acoustics, Sp
ech and Signal Processin
g '86, pp. 2643-2646, 1986 An object of the present invention is to significantly reduce the labor of collecting learning speech, which is a major problem when trying to obtain high recognition performance for speech of various utterance styles. HMM
Speech that was not included in the training speech using limited training speech and computation time, eliminating the need for enormous computation time, such as re-estimation of parameters and codebook redesign. Finding the likelihood of an acoustic model that can achieve high recognition performance even for speech of a style
It is to provide a method for optimizing .

[0006]

According to the present invention, for each recognition target category, n (n is an integer of 2 or more) learning voice sets having different statistical properties are used to correspond to each category. The n acoustic models are created, and the n acoustic models are weighted by coupling coefficients and combined to form one acoustic model for the one recognition target category.

In order to further increase the recognition rate, the coupling coefficient is adapted by learning using a part of the speech to be recognized. This learning is performed by learning using a part of the learning voice, evaluating the remaining learning voice, repeating the same operation by replacing a part of the learning voice, and obtaining an optimum coupling coefficient. In this way, a model having a high recognition performance for the recognition target speech can be constituted by a fraction to one-tenth of the learning speech and the calculation time by the conventional method. Compared to the prior art, (1) no re-learning of the HMM itself is required, (2) no re-design of the codebook is required,
(3) There is an advantage in that a learning voice having the same statistical properties as the recognition target is required to be only about one tenth to several tenths of the conventional method.

[0008]

【Example】

Embodiment 1 Hereinafter, as one embodiment of the present invention, two acoustic models created from learning speech sets having different statistical properties will be described.
HMMs learned from voices uttered separately for each word
And HM learned with voices uttered in sections
A case in which a model adapted to a freely uttered continuous voice using M and a model configured will be described with reference to the drawings.

FIG. 1 shows a speech recognition apparatus to which the present invention is applied. The analog audio signal A is converted into a digital audio signal B by the audio input unit 1, and the digital audio signal B is used to convert acoustic features (for example, cepstrum, Δ cepstrum,
C is extracted by the acoustic feature quantity extraction unit 2.
The coupling coefficient, the likelihood of the model, and the like are calculated by the calculation unit 3,
HMM parameters, coupling coefficients, and the like are stored in the memory 4, and the calculation unit 3 outputs a recognition result E. The acoustic feature quantity extraction unit 2 may be realized by hardware or may be realized by software. In the case of realization by software, it can be realized by the arithmetic unit 3 if the arithmetic capability of the arithmetic unit 3 is sufficient.

FIG. 2 shows an acoustic model in which these two HMMs are combined, and shows a calculation procedure of likelihood calculation of the model. The calculation shown in this figure is performed in the calculation unit 3 of FIG. A combination model is provided for each recognition target category (vocabulary). The acoustic feature value C of the freely uttered continuous speech to be recognized is calculated by calculating the likelihood of the HMM (model 1) for the vocabulary learned from the uttered speech divided into words 5 and similarly for each phrase. HMM is a calculation 6 likelihood of (model 2) made for that vocabulary learned speech uttered separated, the model 1, model 2
For each calculation result of the likelihood observed from, the coupling coefficient λ ₁ ,
lambda ₂ multiplication 7,8 is made, respectively, of these models 1,
The addition 9 of the value obtained by multiplying each likelihood of the model 2 by the coupling coefficient is performed, and the added value is the likelihood D of the combination model with respect to the input speech. That is, this combination model is one acoustic model for this vocabulary, and the likelihood P (x) from which the acoustic feature vector x is output from this acoustic model is the output of the acoustic feature vector x from model 1 and model 2. Let P ₁ (x) and P ₂ (x) be the likelihoods
Then, it is expressed by equation (1). That is, the model 1 and the model 2 are linearly combined.

[0011] _{P (x) = λ 1 P} 1 (x) + λ 2 P 2 (x) (1) _{Here, λ 1 + λ 2 = 1} (1) the coupling coefficient of the equation lambda _1, lambda ₂ is the model 1, Although it may be fixedly determined in advance from the second property, it may be determined so as to be suitable for the recognition target by learning using a continuous utterance learning voice to be recognized. The adaptation is obtained, for example, by repeatedly calculating the equations (2) and (3).

C _j = {λ _j P _j (x _i ) / P (x _i )} (j = 1,2; i = 1,2,3,..., N) (2) λ ′ _j = c _j / Σc _j (3) where N is the number of tokens of the vocabulary in the adaptive learning speech. This is divided into the learning data to two, to learn the model in half, the rest of the data, to estimate the λ "held
-Out-interpolation ". Not limited to this method, for example, " deleted
Λ may be estimated using “−interpolation”. In this case, the λ is obtained by repeating re-estimation using the recurrence formulas (2) ′ and (3) ′.

C _j = Σ ｛λ _j P ⁱ _j (x _i ) / P ⁱ (x _i )｝ (2) ′ λ ′ _j = c _j / jc _j (3) ′ P ⁱ _j (x _i ), P ⁱ (x _i ) is the likelihood of x _i by the model learned except for the i-th data x _i . Substituting λ ′ into the expressions (1) and (2) ′ and repeating re-estimation until the values converge.

FIG. 3 shows a method of judging a recognition result at the time of recognition. Here, as an example, a case where the number of recognized words is five is shown. It indicates that the likelihood for the acoustic feature quantity C of the input speech acoustic model 11 ₁ to 11 ₅ shown in FIG. 3 vocabulary 1 vocabulary 5 becomes D ₁ to D _5, respectively. At the time of recognition, the vocabulary having the maximum of the likelihoods D _{1 to} D ₅ is set as a recognition result, or as a recognition result candidate in the order of higher likelihood.

By using the method of this embodiment, a model trained by using the voice uttered by separating each word and a model learned by using the voice uttered by separating each phrase are linearly combined, Further, FIG. 4 shows the result of a phoneme recognition experiment of 18 consonants in Japanese with a model adapted to the continuously uttered recognition target speech. The collective learning model is a result of using a model learned by simultaneously using word utterance, phrase utterance, and continuous utterance, and the combination model is a model of the above-described embodiment, that is, a model trained by word utterance and a model learned by phrase utterance. The results obtained by using a model obtained by learning the coupling coefficients of a model obtained by linearly combining the models obtained using continuous uttered voices. In the case of (3) without learning, the coupling coefficients are learned and adapted using continuous uttered voices. If not.
(1) When 20 sentences of continuous speech are used in addition to the words and phrases, and (2) When 10 sentences of continuous speech are used in addition to the words and phrases. (3) shows a case where only word utterance and phrase utterance are used, and continuous utterance is not used.
(A), (b), and (c) show types of evaluation voice. From the results shown in FIG. 4, (1), (2), (3)
In all cases, the combination model, that is, the embodiment of the present invention, has a higher recognition rate for continuous utterance speech, and particularly has a higher recognition rate without performing the adaptation learning of the coupling coefficient. To demonstrate the effectiveness of the technique of the present invention. Second Embodiment Next, as a second embodiment of the present invention, a context-independent HMM and a context-dependent HMM are used as models using learning speech sets having different statistical properties, and an acoustic model is uttered by combining these. A case where the system is made robust against a change in style will be described. For example,
The acoustic properties (or acoustic aspects) of the phoneme / a / differ depending on what the preceding and following phonemes are. When a different HMM is provided depending on the difference between the preceding and succeeding phonemes, the HMM is called a context-dependent HMM. For example, / akai /
The first / a / in the word is at the beginning of the word and immediately after / k /
It is. The second / a / is immediately before / k / and immediately after / i /
It is. The first / a / is represented by an HMM # -ak, and the second / a / is represented by an HMM k-a-i. When only / is used as a learning speech, these HMMs are called context-dependent HMMs. Triphone-base is a case where the context of a context-dependent HMM is considered one phoneme before and after.
Say A case in which one of the front and the back is taken into account is called a biphone-base. The more phonemes you choose before and after, the more restrictive the acoustic environmental conditions will be and the more detailed your model will be.
The frequency of appearance decreases, and it becomes difficult to collect learning sounds. On the other hand, when all the / a / are expressed by the HMM of / a / regardless of the phonemes before and after, this HMM is called a context-independent HMM. In the case of a context-independent HMM, / a of various phonemic environments
Since / is used as the learning voice, the variance of the acoustic properties of the learning voice is large, but the frequency of appearance is high, so that many learning voices can be used.

FIG. 5 shows a process for calculating the likelihood of a model obtained by linearly combining a context-independent HMM and a context-dependent HMM. Model 12 ₁ in the context independent HMM, model 12 _2-model 12 _M is (M
-1) context-dependent HMMs. For example, M
= Time of 3, the model 12 ₂ biphone-base
Of the model 12 ₃ and context-dependent HMM for triphone-base. Further, more detailed context-dependent HMMs such as quadraphone base (quadraphone-base 4 chain) and quadraphone base (quintphone-base 5 chain) may be considered. As in Example 1, the likelihood P of acoustic feature vectors x of a combination model is output (x) is the acoustic feature vector x is output from the model 12 ₁ to model 12 _M, respectively likelihood P ₁ (x ) to P and _M (x), when the coupling coefficient to be multiplied by their respective 13 ₁ to 13 _M and lambda ₁ to [lambda] _M, is expressed by the following equation.

[0017] _{P (x) = λ 1 P} 1 (x) + λ 2 P 2 (x) + ... + λ M P M (x) (4) _{Here, λ 1 + λ 2 + ...} + λ M = 1 (4) Λ ₁ to λ _M in the equations are obtained by repeatedly calculating the following equations (5) and (6). c _j = Σ ｛λ _j P _j (x _i ) / P (x _i )｝ (5) λ ′ _j = c _j / Σc _j (6) Here, assuming M = 3, P ₁ (x _i ) is A context-independent HMM, P ₂ (x _i ) is a biphone-base context-dependent HMM, and P ₃ (x _i ) is a triphon.
Assuming that the e-base context-dependent HMM is used, the likelihood P (x _i ) for the observation vector x _i of a model combining these is expressed by equation (7).

P (x _i ) = λ ₁ P ₁ (x _i ) + λ ₂ P ₂ (x _i ) + λ ₃ P ₃ (x _i ) (7) where λ ₁ + λ ₂ + λ ₃ = 1 By learning λ ₁ , λ ₂ , and λ ₃ using a part of the speech to be recognized, the speech is adapted to the speech to be recognized. Embodiment 3 HMMs learned for each speaker are used as HMMs obtained from learning speech sets having different statistical properties, and these can be combined to adapt to a new speaker's utterance. The models learned with the voices of the different speakers are combined as shown in FIG. 5, and the coupling coefficient is adaptively learned with the voice of the recognition target speaker. The method of learning the coupling coefficient in the adaptive learning is the same as the method described in the second embodiment. Embodiment 4 In an HMM-based speech recognition method of learning an acoustic model using a learning speech, the acoustic model is easily affected by a difference in a sound collection system between the learning speech and the actual speech to be recognized. In order to construct a recognition system that is highly resistant to differences in sound collection systems,
According to the method of the present invention, models learned individually by using sounds collected by a plurality of different sound collection systems as learning sound sets having different statistical properties are linearly combined, and their coupling coefficients are determined by the sound collection system of the recognition target sound. By performing the adaptive learning using the voice collected by the above, the influence of the voice collection system can be reduced.

[0019]

According to the present invention, HMM acoustic models are created from learning speech sets having different statistical properties, and these acoustic models are weighted by the coupling coefficient for the same category and combined to form one acoustic model for that category. By doing so, the recognition rate can be increased. In particular, by adapting the coupling coefficient with the recognition target speech, an HMM acoustic model adapted to the recognition target speech having a statistical property different from that of the learning speech can be configured. In the adaptive learning, since only the coupling coefficient is learned without learning the parameters of the HMM itself, the number of parameters to be learned is very small. Therefore, the adaptation can be performed with a small learning voice and a short calculation time.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a general configuration of a speech recognition system.

FIG. 2 is a diagram showing a state of a linear combination of two models in an example of the present invention.

FIG. 3 is a view for explaining a method of determining a recognition result at the time of recognition using the combined model obtained in FIG. 2;

FIG. 4 is a diagram showing recognition results for 18 Japanese consonants.

FIG. 5 is a diagram showing a state of linear combination of a plurality of models according to the present invention in a general case.

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-61-183696 (JP, A) JP-A-62-5300 (JP, A) JP-A-61-180298 (JP, A) JP-A-61-180298 121093 (JP, A) IEICE Technical Report (SP 89 46-53) pp. 17-24 Examination of Japanese Phoneme Recognition by Continuous Output Distribution HMM (58) Fields Investigated (Int. Cl. ⁷ , DB G10L 15/14 G10L 15/06

Claims (2)

(57) [Claims] 1. Statistical modeling of acoustic features of speech
The likelihood of the input acoustic model for the input speech
In each of the categories to be recognized,
(N is an integer of 2 or more) learning sounds with different statistical properties
N individual acoustic models created using sets
Of each of the above-mentioned input voices is obtained.
The individual likelihood is determined in advance according to the properties of each of the above acoustic models.
By weighting with the determined coupling coefficient and combining
Acoustic model characterized by finding desired likelihood
A method to find the likelihood of the input speech . 2. The likelihood of an acoustic model for an input speech according to claim 1, wherein the coupling coefficient is adapted by learning using a part of the speech to be recognized.
Mel method.