JPH07239695A - Learning method of hidden markov model - Google Patents

Abstract

PURPOSE:To permit learning of phoneme HMMs(Hidden Markov Models) with high accuracy by eliminating the inconsistency of the voice data and phoneme notation according to a difference in nasalization of 'GA' line voices or long vowels at the time of learning the phoneme HMMs from sentence voices by a consolidated learning method. CONSTITUTION:A sentence HMM is first formed by using the phoneme notation information belonging to the inputted sentence voices and substituting /g/ and /ng/ at the point where the 'GA' line syllable exists while referencing a phoneme HMM dictionary 4 for recognition in a step 5 at the time of learning the phoneme HMMs by using the sentence voices. The sentence HMM and the input voices are collated and the syllable notation is determined by calculating tolerance in a step 6 and thereafter, the phoneme HMMs are connected to form the sentence HMM in a step 7. The learning of the sentence HMM is executed in a step 8 and the sentence HMM is decomposed into the phoneme HMMs in a step 9 and thereafter, whether the phoneme HMMs converge or not is decided in a step 12. The leaning is returned to the learning of the sentence HMM of the step 8 in a step 11 and the learning processing and the separating processing are repeated in case of non-conversion.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a learning method for a Hidden Markov Model (hereinafter referred to as HMM) in speech recognition.

[0002]

2. Description of the Related Art Conventionally, as a technique in such a field,
For example, some documents were described in the following documents. Reference 1: The Bell SystemTechnical Journal, 62 [4],
(1983-4), American Telegraph Company,
(US), SE Levinson (SELevinson), ER Rabiner (LR Rabiner), MM Sondhi (MM Sondhi), `` An Introduction to the App
lication of the Theory of Probabilistic Function o
fa Markov Process to AutomaticSpeech Recognitio
n "P. 1035-1074 Reference 2; Seiichi Nakagawa, "Speech Recognition by Stochastic Model" (SHO 63-7), IEICE, P55-61. As a speech recognition technology, from classical pattern matching method to statistically in recent years. However, the latter is becoming mainstream. As a statistical method, a Markov model having a stochastic finite state (hereinafter referred to as HMM) has been proposed. Generally, the HMM is a transition between states and a plurality of states (for example, voice characteristics). Furthermore, the HMM has a transition probability that represents a transition between states and an output probability that outputs a label (a typical feature parameter of speech, usually several tens to several thousands) associated with the transition. ing. A speech recognition method using such an HMM is described in the above document, and an example of the word speech recognition is shown in FIG.

FIG. 2 shows the word HM used in the speech recognition method.
It is a figure which shows M structure. S ₁ , s ₂ , s ₃ , s _{4 in} FIG.
Represents a state such as a voice feature in the HMM. a ₁₁ , a
₁₂ , a ₂₂ , a ₂₃ , a ₃₃ , a ₃₄ , a ₄₄ , a ₄₅ are state transition probabilities, b ₁ (k), b ₂ (k), b ₃ (k), b ₄ (k) are label outputs Represents In the HMM, the state transition probability a _ij (i = 1, ..., 4, j =
When the state transition is performed in 1, ..., 5), the label output probability b _j
The label is output at (k). Using the HMM for the spoken word, the HMM is learned so as to output the label string of the word with the highest probability. Next, the label string of the uttered unknown word is input, and the word HMM that gives the highest output probability is used as the recognition result. By replacing a word with a sentence, it is possible to recognize a voice uttered for each sentence in the same manner. In this type of speech recognition method, an HMM is given to a spoken word or a sentence itself to learn, and a recognition result is judged based on a likelihood (that is, an output probability of a label string).
Such a word or sentence HMM guarantees excellent recognition accuracy, but has a drawback that a huge amount of learning data is required due to an increase in the number of recognized vocabulary words, and speech other than the learning target word cannot be recognized at all. is there.

On the other hand, in phonetics, words or sentences are usually represented by a series of vocal elements called phonemes. Therefore, an HMM is prepared for each phoneme, these HMMs are connected to generate a word or sentence HMM, an HMM is prepared for each phoneme, these HMMs are connected to generate a word or sentence HMM, and word recognition is performed. There is a way to do. In particular, when recognizing sentence voices, it is difficult to prepare a large amount of sentence voices, and thus it is almost impossible to learn HMMs of all sentences to be recognized. Therefore, in the case of a sentence voice, the sentence H is obtained from the phoneme HMM.
Generating an MM is a realistic means. In order to learn phonemes, it is necessary to prepare information indicating the section in which each phoneme exists in the learning data, that is, label information. However, as for the labeling work, the automatic work by the computer does not provide satisfactory accuracy, and most of the labeling work is done manually. Therefore, a learning method that does not require label information has been proposed. In this method, first, an initial model of the phoneme HMM is prepared. Then, the sentence HMM is constructed by connecting the above initial model of the phoneme HMM to the learning data of the sentence utterance whose utterance content is known and which is not labeled.
Learn M with the learning sentence vocalization data. In this case, the learning process can be realized if the beginning and end of the sentence are known. Furthermore, these sentence HMMs are decomposed by the reverse procedure of concatenation, and the phoneme H
Generate MM. In order to improve the learning accuracy, the phoneme HMM with high accuracy is generated by repeating the above-described connection learning and decomposition generation. Naturally, this connection learning method can also be applied to word speech.

[0005]

However, the conventional phoneme HMM connection learning method has the following problems. (1) When the utterers utter a sentence, the "ga" line and the "ga" line sound (hereinafter simply referred to as the "ga" line) are different because the origins of each speaker and the educational background are different. Whether it is nasalized depends on the speaker. For example, "Kokugo (Japanese)" is / kokugo / and / ko
There are two types of vocalizations, such as kungo /. When learning the HMM, ignoring this difference and learning by concatenating the phoneme HMMs according to the phoneme notation (or nasalized) phoneme notation, speech data of different properties are assigned to the same HMM model, and the phoneme The HMM accuracy is unavoidably reduced. Therefore, it is necessary to detect the difference in utterance of this "ga" line. Conventionally, detection of the nasalization of this "ga" line has to be done by human hearing, which requires a large amount of time and labor. (2) When a utterer utters a sentence voice, the method of uttering a long sound varies depending on the utterer because the origin and educational background of each utterer are different. For example, "Tokyo (Tokyo)" is / tokyo /// tokyo
There are four possible pronunciations, such as /, / tokyo /, and / tokyo /. When learning the HMM, ignore this difference and uniformly use the romanization (ie / touki
When phoneme HMMs are connected and learned in accordance with ou /), voice data having different properties are assigned to the same HMM model, and the phoneme HMM accuracy is unavoidably deteriorated. Therefore, it is necessary to detect the difference in utterance of long sound. Conventionally, the detection of the difference in long-sounding utterance must be done by human hearing.
It takes a lot of time and effort. In this way, the phoneme HMM is changed according to the nasalization of the “ga” line and the utterance of different long sounds.
In order to avoid the decrease in the accuracy of the phoneme HMM, the detection of nasalization of "ga" lines and the difference in the utterance of the long sound must be done by human hearing, which requires a lot of time and labor. There was a problem that it was necessary.

[0006]

In order to solve the above-mentioned problems, the first invention uses a phoneme HMM by using continuous speech data.
When learning, the sentence HMM is constructed by connecting the initial models of the phoneme HMM. Then, a learning process for learning the sentence HMM, and a learning result after the learning process for the phoneme H
Decomposition processing for decomposing into MM and the decomposed phoneme HMM
In the HMM learning method of learning the phoneme HMM by repeating the learning process, the decomposition process, and the concatenation process by using the concatenation process for reconstructing the sentence HMM and the sentence HMM, the following means are taken. When the “ga” line syllable or the “ga” line sound included in the learning sentence voice data is nasalized is detected by a voice recognition method, and when the sentence HMM is generated by connecting the phoneme HMMs, the recognition is performed. According to the result, the corresponding phoneme HMMs are connected and learned, and the phoneme HMMs are learned. The second invention adopts the following means in the HMM learning method similar to the first invention.
A difference in utterance of long sounds included in the sentence sentence learning data is detected by a speech recognition method, and the phoneme HMMs are connected to each other to connect the sentence H.
When the MM is generated, the phoneme HMM corresponding to the utterance of the long sound is connected and learned in accordance with the recognition result, and the phoneme HMM is learned.

[0007]

According to the first aspect of the present invention, since the HMM is configured as described above, if it is assumed that there are N "ga" syllables or "ga" gospels included in the learning sentence voice data, The number of sentence HMMs in which the "ga" line syllable or the "ga" line sound can be nasalized is 2 ^N , and the sentence sentence HMM of these 2 ^{N and} the sentence sentence voice data for learning are compared to obtain the sentence sentence voice data for learning. A voice recognition method detects whether or not the included “ga” syllable or “ga” syllable has been nasalized. As described above, before performing the connection learning, it is accurately detected whether the “ga” line syllable or the “ga” line sound included in the learning sentence voice data is nasalized by the voice recognition method, and the corresponding HMM is connected. Since the learning is performed in this manner, it is possible to recognize whether the “ga” syllable or the “ga” syllable has been nasalized, and the phoneme HMM can be learned with high accuracy. According to the second aspect of the invention, since the HMM is configured as described above, a long-sound is obtained by matching a plurality of sentence HMMs and learning-sentence voice data that are possible due to the difference in utterance of the long-sound included in the learning-sentence voice data. The difference in the utterances of the voices is detected by the voice recognition method. As described above, before performing the connected learning, the difference in the long voice utterance included in the learning sentence voice data is accurately recognized by the voice recognition method, and the HMM corresponding to the long voice utterance included in the learning sentence voice data is connected. Since the learning is performed in this way, the difference in the utterance of the long sound can be recognized, and the learning of the phoneme HMM with high accuracy can be performed. Therefore, the above problem can be solved.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment FIG. 1 is a flowchart of the processing contents of an HMM learning method showing a first embodiment of the present invention. The HMM of the first embodiment will be described with reference to this figure. Explain the learning method. In the learning method of the HMM according to the first embodiment, for example, the steps 1 to 1 in FIG.
The process of 13 is performed. First, when learning is started in step 1 of FIG. 1, a voice signal (for example, sentence voice) of learning data is input in step 2, and step 3
Proceed to the pre-processing. In the pre-processing of step 3, for example, the input analog audio signal is converted into a digital signal by analog / digital conversion (hereinafter referred to as A / D conversion), and LPC (Linear Predictive Coding) analysis is performed. The voice feature parameter is extracted by extracting the LPC cepstrum or the like, and the process proceeds to step 5.
In step 5, the phoneme notation information attached to the input sentence voice is used to refer to the recognition phoneme HMM dictionary 4 prepared in advance to detect a position where a "ga" line syllable exists and / Substitute g / and / ng / to generate the sentence HMM,
Go to step 6. In step 6, the sentence HMM generated in step 5 is collated with the input voice, the likelihood is calculated,
The sentence HMM giving the highest output probability is passed to the next step 7 as the correct phoneme notation. By the above steps 1 to 6, it is recognized whether or not the "ga" line syllable included in the learning sentence voice data is nasalized.

[0009] For example, if the sentence "Manipulate kokugo (learn Japanese)" is input, in step 5, the following two sentence HMMs are generated. (1) / kokugowomanabu / (2) / kokungoumanabu / In step 6, these two sentence HMMs are collated with the input speech to calculate the likelihood. Here, the (2) th H
If MM gives a higher output probability, then / kokungowomanabu / is passed to the next step 7 as the correct phoneme notation. In step 7, while referring to the phoneme notation determined in step 6 and the phoneme dictionary 10, the phoneme HMMs are connected and the sentence HMM is connected.
Is generated and the generation result is sent to step 8. In step 8, the sentence HMM parameters are estimated using the input learning voice. For this estimation, for example, the Baum-Welch (BW) algorithm described in Document 2 is used. In this B-W algorithm, for example, the observation label series O
_{= O 1, o 2, ...} , o T and state series _{I = i 1, i 2,} ..., i
_{For T} , a forward variable α _t (i) and a backward variable β _t (i) as defined by the following equation (1) are defined. _{α t (i) = Pr (} o 1, o 2, ..., o T, i t = s i) β t (i) = Pr (o t + 1, o t + 2, ..., o T, i t = s _i ) (1) Then, the state transition probability a _ij and the label output probability b _j (k) are estimated as in the following expression (2).

[0010]

[Equation 1] After learning the sentence HMM in this way, in step 9, the sentence HMM is decomposed into phoneme HMMs, and the corrected phoneme HMMs are stored in the phoneme dictionary 10. Whether or not this phoneme HMM has converged is checked in step 12, and if it converges (that is, if the difference between the previous value and the current value of the phoneme HMM parameter is sufficiently small), the learning ends in step 13. To do. On the other hand, if the result of the check in step 12 is that they do not converge, in step 11, the phoneme HMMs decomposed in step 9 are concatenated to reconstruct a sentence HMM,
In step 11, the process returns to the learning of the sentence HMM in step 8, and the learning process and the decomposition process described above are repeated.

As described above, the first embodiment has the following advantages. When learning a phoneme HMM using sentence speech, in step 5, / g / and / ng / are substituted into the place where the "ga" line syllable exists to generate a sentence HMM, and in step 6, these sentence HMMs are input. Match the voice, calculate the likelihood,
Since it is detected whether or not the "ga" line included in the learning voice data is nasalized, it is possible to accurately perform the connected learning. Therefore, it is possible to eliminate the inconsistency between the phonetic notation and the voice data that accompanies the nasalization of the “ga” line voice, and it is possible to perform highly accurate HMM learning.

Second Embodiment FIG. 3 is a flowchart of the processing contents of the learning method of the HMM showing the second embodiment of the present invention. The first embodiment is for detecting whether or not the "ga" line included in the learning sentence voice data is nasalized, whereas the second embodiment is included in the learning sentence voice data. It detects long sound. With reference to this figure, the HM of the second embodiment
The M learning method will be described. In the HMM learning method of the second embodiment, for example, the processing of steps 21 to 33 of FIG. 1 is executed using a program-controlled computer. First, learning is started in step 21 of FIG. Then, in step 22, a voice signal of the learning data (for example,
(Sentence voice) is input, and the process proceeds to the preprocessing of step 23. In the preprocessing of step 23, for example, the input analog voice signal is converted into a digital signal by A / D conversion, the voice characteristic parameter is extracted by extraction of the LPC cepstrum by LPC analysis, and the process proceeds to step 25. In step 25, the phoneme notation information attached to the input sentence speech is used to refer to the recognition phoneme HMM dictionary 24 to detect a part in the long syllable and substitute a phoneme HMM of possible utterance therein. The sentence HMM is generated, and the process proceeds to step 26. In step 26, the sentence HMM generated in step 25 is collated with the input voice, the likelihood is calculated, and the sentence HMM giving the largest output probability is passed to the next step 27 as the correct phoneme notation. Steps 21 to 26 above
By the processes up to, the difference in the utterance of the long sound included in the learning sentence voice data is recognized.

For example, if the sentence "Tokyo ni Iku (go to Tokyo)" is input, in step 25, the following four sentence HMMs are generated. (1) / tokyouniiku / (2) / tokyouniuk / (3) / tokyouniuk / (4) / tokyooniiku / In step 26, the four sentence HMMs are compared with the input speech, and the likelihood is calculated, and in step 27. move on. In step 26, the HMM corresponding to the pronunciation of the long sound included in the sentence voice input is detected. Here, if
Assuming that the (4) th HMM gives a larger output probability, then / tokyokyoniku / is passed to the next step 27 as the correct phoneme notation. In step 27, referring to the phoneme notation determined in step 26 and the phoneme dictionary 30, the phoneme HMMs are concatenated to generate a sentence HMM, and the generation result is sent to step 28. In step 28, the sentence HMM is input using the input learning voice.
Estimate the parameters. For this estimation, for example, B-W
Using an algorithm, for example, observation label series O = o
_{For 1} , ₁ , o ₂ , ..., o _T and state series I = i ₁ , i ₂ , ..., i _T , forward variable α _t (i) and backward variable β _t (i) as shown in equation (1). Is defined. Then, the state transition probability a _ij
And the label output probability b _j (k) are estimated as in Expression (2).

After learning the sentence HMM in this way, in step 29, the sentence HMM is decomposed into phoneme HMMs.
The corrected phoneme HMM is stored in the phoneme dictionary 30. Whether or not the phoneme HMM has converged is checked in step 32, and if converged (that is, if the difference between the previous value and the current value of the phoneme HMM parameter is sufficiently small), the learning ends in step 33. To do. On the other hand, as a result of the check in step 32, if the results do not converge, in step 31, the phoneme HMMs decomposed in step 29 are concatenated to reconstruct a sentence HMM, and in step 31, step 2
Returning to the learning of the sentence HMM of No. 8, the learning process and the decomposition process described above are repeated. As described above, the second embodiment has the following advantages. When learning a phoneme HMM using sentence speech, a sentence HMM is generated by substituting a phoneme HMM of a possible utterance into a position in a long syllable in step 25 to generate a sentence HMM, and in step 26, the sentence HMM and the input speech are collated, Since the HMM corresponding to the pronunciation of the long sound included in the input sentence voice is detected, the connection learning can be accurately performed. Therefore, it is possible to eliminate the inconsistency between the voice data and the phoneme notation associated with the difference in long-sound utterance, and to perform highly accurate HMM learning. The present invention is not limited to the above embodiment, and various modifications can be made.
The following are examples of such modifications. Although sentence speech has been described as an example, it can be applied to word speech and phrase speech.

[0015]

As described in detail above, according to the first aspect of the invention, first, "ga" included in the learning sentence voice data is included.
A syllable and a “ga” are detected by a speech recognition method to determine whether the syllable is nasalized, and when the phoneme Hidden-Markov model is concatenated to generate the sentence Hidden-Markov model, the detected “ga” is detected. Since the Hidden-Markov model corresponding to the "ga" and "ga" gospels is substituted in the places where the syllable and "ga" gospel are present, it is possible to perform accurate connected learning. it can. Therefore, it is possible to eliminate the inconsistency between the phoneme notation and the voice data that accompanies the nasalization of the “ga” line voice, and it is possible to perform highly accurate HMM learning. According to the second aspect of the present invention, the difference in utterance of long sounds included in the learning sentence voice data is detected by the voice recognition method, and the phoneme Hidden Markov
When connecting the models to generate the sentence Hidden-Markov model, the Hidden-Markov model corresponding to the utterance of the long note is connected and learned in accordance with the recognition result. It can be carried out. Therefore, it is possible to eliminate the inconsistency between the voice data and the phoneme notation associated with the difference in long-sound utterance, and to perform highly accurate HMM learning.

[Brief description of drawings]

FIG. 1 is a flowchart of processing contents of an HMM learning method according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a structural example of a word HMM used in a conventional speech recognition method.

FIG. 3 is a flowchart of processing contents of an HMM learning method according to a second embodiment of the present invention.

[Explanation of symbols]

4,24 Phoneme HMM dictionary for recognition 5 Sentence H in consideration of nasalization of "ga" line syllable
MM generation process 6,26 Phoneme notation determination process by HMM likelihood calculation 7,27 Sentence HMM construction process by concatenation of phoneme HMMs 8,28 Learning HMM sentence BW algorithm process 9,29 Sentence HMM decomposition process for sentence HMMs 10 , 30 Phoneme HMM dictionary 11, 31 Reconstruction process of sentence HMM 12, 32 Convergence determination process of phoneme HMM 25 Sentence HM considering fluctuation of vocalization of long sound
M generation process

Claims (2)

[Claims] 1. When learning a phoneme Hidden-Markov model using continuous speech data, an initial model of the phoneme Hidden-Markov model is connected to construct a sentence-Hidden-Markov model, and the sentence-Hidden-Markov model is constructed. A learning process for learning a model, a decomposition process for decomposing the learning result into a phoneme Hidden Markov model after the learning process, and a sentence Hidden Markov model by reconstructing the decomposed phoneme Hidden Markov model In the learning method of the hidden Markov model for learning the phoneme Hidden-Markov model by repeating the learning process, the decomposition process, and the concatenation process using "Ga" line syllable or "ga"
A method of detecting whether or not the syllable is nasalized by a speech recognition method, and connecting the phoneme Hidden-Markov model to generate the sentence Hidden-Markov model, the corresponding Hidden-Markov model according to the recognition result. A learning method for a Hidden-Markov model, which comprises learning by connecting the phonemes to learn the phoneme Hidden-Markov model. 2. When learning a phoneme Hidden Markov model using continuous speech data, a sentence Hidden Markov model is constructed by connecting initial models of the phoneme Hidden Markov model, and the sentence Hidden Markov model is constructed. A learning process for learning a model, a decomposition process for decomposing the learning result into a phoneme Hidden Markov model after the learning process, and a sentence Hidden Markov model by reconstructing the decomposed phoneme Hidden Markov model In the learning method of the hidden Markov model for learning the phoneme Hidden-Markov model by repeating the learning process, the decomposition process, and the concatenation process using The difference in the utterance of the long sound is detected by a speech recognition method, and the phoneme Hidden-Markov model is connected to the sentence. When generating a Don-Markov model, the phoneme Hidden-Markov model corresponding to the vocalization of the long sound is connected and learned in accordance with the recognition result, and the phoneme Hidden-Markov model is learned.・ How to learn Markov model.