JP2738508B2 - Statistical language model creation device and speech recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a statistical language model creating apparatus for automatically creating a statistical language model for speech recognition, and a word input by referring to the statistical language model created by the statistical language model creating apparatus. The present invention relates to a speech recognition device for recognizing an utterance sentence composed of a sequence.

[0002]

2. Description of the Related Art As language models used in speech recognition, syntax models represented by network grammar and context-free grammar, and statistical models represented by bigrams and trigrams are used. . Although the former syntactic language model has been widely studied in the field of natural language processing, it requires a great deal of effort for humans to describe syntax rules based on knowledge about languages.
On the other hand, statistical models such as bigrams and trigrams are widely used as language models in the field of speech recognition because they are simple models, but these models are too simple to express languages. . Therefore, to complement the problems of both models, a network grammar with probability and a context-free grammar with probability have been proposed and studied.

[0003] In the field of speech recognition, a hidden Markov model (hereinafter, referred to as HMM) is often used. As shown in FIG. 2, among the types of HMMs, as shown in FIG. 2, when a word is set as an output symbol in an Ergodic discrete HMM (hereinafter, referred to as E-HMM) in which transition between all states is allowed, Its structure is formally similar to a network grammar description. Therefore, there is a possibility that a stochastic network grammar can be automatically created from a large amount of word string data using a Baum-Welch learning algorithm, for example, using a digital computer.

[0004]

However, when the number of states is increased, the amount of memory and the amount of calculation of a digital computer for creating a network grammar with probability using the learning algorithm of Balm-Welch increase exponentially. In practice, calculation becomes impossible. According to the inventor's trial calculation, when the number of states exceeds 30, calculation becomes impossible. For this reason, previous studies have found that the number of states is small, and that speech recognition performance and perplexity are worse than bigrams of words (for example, Murase et al., “Context-free grammar and bigram. tr
Study of Language Modeling Using igram ", IEICE Technical Report, SP90-75, NLC90-47,
pp. 49-56, 1990. ).

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems, to automatically generate a statistical language model using a Balm-Welch learning algorithm, and to obtain a higher speech recognition rate than the conventional example. It is an object of the present invention to provide a statistical language model creation device capable of performing the above-mentioned operations and a speech recognition device using the same.

[0006]

According to a first aspect of the present invention, there is provided a statistical language model creating apparatus which stores a statistical language model in the course of learning and a learning text data input thereto. A statistical language model for speech recognition or morphological analysis, which is an Ergodic hidden Markov model of a predetermined number of states including parameters of initial state probability, state transition probability, and symbol output probability, a learning algorithm of Balm-Welch A statistical language model creating device comprising: a learning unit that learns and uses the learning unit. The control unit estimates and calculates the parameter using the Balm-Welch learning algorithm, and the symbol output probability is a predetermined value. Is smaller than the threshold value, the symbol output probability is set to 0, and the transition of the set symbol output probability is set. Deleting means for deleting the previous state from the storage device; and, when learning the parameters using the Balm-Welch learning algorithm, sequentially setting the number of states of the Ergodic hidden Markov model to twice. Parameter calculation means for calculating the parameters of the Ergodic hidden Markov model when the number of states is large.

According to a second aspect of the present invention, in the statistical language model creating apparatus according to the first aspect, the parameter calculating means calculates the first state number from the first state number. When calculating a parameter of the second number of states that is twice the number of the two states, two parameters having the same initial state probability as the first number of states at the first number of states are calculated for each of the two states; By multiplying the two calculated initial parameters by 0.5 for normalization,
First calculating means for calculating initial parameters of initial state probabilities at the second number of states, and four calculating means for every four states having initial parameters of the same state transition probability at the first number of states Second calculating means for calculating the initial parameters of the state transition probability at the second number of states by multiplying the calculated four initial parameters by 0.5 for normalization And for each of the four states,
Calculate four parameters having initial parameters of the same symbol output probability at the number of states, multiply each of the calculated initial parameters by a different random number generated by a predetermined method, and calculate the multiplication result. And a third calculating means for calculating a parameter of the symbol output probability at the second number of states by normalizing the parameter so that the sum of the symbol output probabilities at the second number of states becomes 1. It is characterized by having.

According to a third aspect of the present invention, there is provided a speech recognition apparatus for creating a statistical language model for speech recognition,
Voice recognition means for recognizing the uttered voice by referring to the statistical language model created by the statistical language model creating device based on the speech signal of the uttered speech composed of the input word string It is characterized by.

[0009]

In the statistical language model creating apparatus according to the first aspect of the present invention, when the parameter is estimated and calculated using the Balm-Welch learning algorithm, the deleting means outputs the symbol. When the probability is smaller than a predetermined threshold, the symbol output probability is set to 0, and the state to which the set symbol output probability transitions is deleted from the storage device. When learning the parameters using the Balm-Welch learning algorithm, the parameter calculation means sets the number of states of the ergodic hidden Markov model to twice in succession to obtain a larger number of states. Compute the parameters of the Ergodic Hidden Markov Model. Therefore, while the amount of memory of the storage device and the amount of calculation of the control unit can be significantly reduced by the reduction unit, a larger number of states can be obtained by the parameter calculation.
-An HMM can be created. That is, E-HM
A statistical language model including M can be created by learning using a Balm-Welch learning algorithm.

Further, in the statistical language model creating apparatus according to the second aspect, the parameter calculating means converts a parameter of a second state number, which is twice the first state number, from the first state number. When performing the calculation, the apparatus preferably includes the first to third calculation means. Here, the first calculation means calculates, for each of the two states, two parameters having the same initial state probability at the first number of states and the calculated two initial parameters. Is multiplied by 0.5 for normalization to calculate an initial parameter of the initial state probability at the second number of states. Then
The second calculation means calculates, for each of the four states, four parameters having the same initial parameter of the same state transition probability as the first number of states, and normalizes to the calculated four initial parameters. The initial parameter of the state transition probability at the second number of states is calculated by multiplying by 0.5. Further, the third calculating means calculates, for each of the four states, four parameters having the same initial parameters of the same symbol output probability at the first number of states, and calculates the four initial parameters. Each of them is multiplied by a different random number generated by a predetermined method, and the result of the multiplication is referred to as a second random number.
By normalizing so that the sum of the symbol output probabilities at the number of states is 1, the parameter of the symbol output probability at the second number of states is calculated.

Further, in the speech recognition device according to the third aspect, the statistical language model device creates a statistical language model for speech recognition. Then, the speech recognition unit performs speech recognition on the uttered speech with reference to the statistical language model created by the statistical language model creating device based on the speech signal of the uttered speech composed of the input word string. .

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A speech recognition apparatus according to an embodiment of the present invention will be described below with reference to the drawings. The speech recognition apparatus of the present embodiment shown in FIG. 1 particularly uses the Balm-Welch learning algorithm based on learning text data composed of word strings stored in the learning text data memory 10,
In addition, by executing a language creation model process described below using a model memory 12 connected thereto and used as a working area for temporarily storing a statistical language model during learning, a statistical language model of the E-HMM is obtained. And a language model creating unit 11 that creates the statistical language model and stores it in the statistical language model memory 7. Here, the speech recognition apparatus performs a phoneme matching process by referring to a microphone 1, a feature extraction unit 2, a buffer memory 3, and an HMM which is an acoustic model in an HMM memory 5 based on input uttered speech data. And outputs a phoneme data, and a phoneme matching unit 4 based on the phoneme data from the phoneme matching unit 4.
ne pass DP (Viterbi search)
h) On which performs speech recognition with reference to the statistical language model in the statistical language model memory 7 using an algorithm
e pass DP speech recognition unit (hereinafter, referred to as speech recognition unit) 6.

First, the language model creation processing executed by the language model creation unit 11 will be described below. First, the learning algorithm of Balm-Welch will be described. The parameters that define the model of the HMM are:
Transition probability a _N between state i and state j when the number of states is _N
(I, j) and symbol output probability b from each state
_N (i, j, k). An important feature of the HMM is that there is an algorithm for estimating and learning them using voice data. A typical example is the learning algorithm of Balm-Welch, also known as forward.
-Also referred to as a backward algorithm. In this algorithm, forward and backward calculations are repeated on a trace, and parameters are updated so as to gradually give a higher probability to learning data. The learning algorithm of Balm-Welch is as follows.

First, for example, a series Y = ｛y ₁ , y ₂ ,..., Y of feature parameters of speech data observed as a model λ.
Given _r }, the probability γ _t (i) of being in or passing through state i at time t is defined by the following _equation (1).

[0015]

Γ _t (i) = P (q _t = i | Y, λ)

Here, q _t indicates the number of the state at the time t, and the probability γ _t (i) of Equation 1 is the partial observation sequence {y ₁ , y ₂ , y during the period from the time 1 to the time t. ₃ ,..., Y _t }, at time t, the forward variable α which is the probability of being in state i
_t (i) and the probability of observing the partial observation sequence {y _{t + 1} , y _{t + 2} , y _{t + 3} ,..., y _r } from time t + 1 to the end in state i at time t Using the variable β _t (i), it can be expressed as in the following Expression 2.

[0017]

(Equation 2)

If the probability of being in state i at time t and being in state j at time t + 1 is ξ _t (i, j),
It is represented by

[0019]

Ξ _t (i, j) = P (q _t = i, q _{t + 1} = j│Y, λ)

From the definition of the forward variable α _t (i) and the backward variable β _t (i), the following _equation 4 is obtained.

[0021]

Ξ _t (i, j) = {α _t (i) a _N (i, j) b _N (i, j,
k) β _{t + 1} (j)} / P (Y│λ)

This equation 4 is rewritten as the following equation 5.

[0023]

(Equation 5)

Here, the probability γ _t (i) and the probability ξ _t (i,
j) has the relationship shown in the following Expression 6.

[0025]

(Equation 6)

As described above, when the observation signal y _t is a discrete code, the initial state probability πh (i), the state transition probability ah _N (i, j) and the symbol output probability bh _N (i,
j, k) can be expressed by the following equations 7, 8, and 9, respectively.

[0027]

Πh (i) = probability of being in state i at time 0 = γ
₁ (i)

(Equation 8)

(Equation 9)

The learning algorithm of Balm-Welch calculates the model λ after re-estimation by performing re-estimation calculation.
h, the probability P (Y│λh) of outputting the sequence Y by the model λ before re-estimation P (Y│λ)
Is guaranteed to be Therefore, by repeating this procedure, a model closer to model λ can be obtained. In this way, by giving an appropriate initial value and repeatedly calculating to increase the probability that the learning text data is observed from the model, it is possible to obtain the HMM parameters based on the learning text data. If a large amount of phoneme pattern data is obtained for each phoneme, HMM parameters that maximize the probability of generating such data can be estimated and calculated using this Balm-Welch learning algorithm. . Furthermore, if a sufficient amount of voice data including all deformations of the phoneme parameters is obtained, a model can be further obtained.

By the way, the parameter λ = (Π, Π) of the transition output type E-HMM of the number of states N (where N is a natural number) used in the language model creation processing of this embodiment.
A, B) define an initial state probability Π, a state transition probability A from state i to state j, and a symbol output probability B from state i to state j as shown in the following equations 10 to 12, respectively. I do.

[0030]

１０ = π _N (i); i = 1, 2,..., N

A = a _N (i, j); i = 1, 2,..., N, j = 1, 2,.

B = b _N (i, j, k); i = 1, 2,..., N, j = 1, 2,..., N, k = 1, 2,.

Here, V is the number of words. The features of the language model creation processing executed by the language model creation unit 11 of the present embodiment are as follows. (Feature 1) The state of small symbol output probability is deleted. That is, when the parameters of the E-HMM are estimated and calculated using the Balm-Welch learning algorithm, the symbol output probability b _N
When (i, j, k) is smaller than a predetermined threshold value, the symbol output probability b _N is set to 0, re-estimation calculation is not performed, and the transition destination state of the symbol output probability b _N is stored in the model memory. 12 is deleted. Thereby, the model memory 1
2 and the calculation amount of the language model creation unit 11 can be significantly reduced. The threshold value is a number exceeding 0 and less than 1, and a preferable value is set to 10 ^−300, and the value of the threshold value is used in a simulation described later in detail. (Feature 2) The number of states is sequentially increased. That is, when re-learning parameters of the E-HMM having a large number of states, a large amount of model memory 12 is required. Therefore, the number of states is sequentially increased. Assuming that the parameters of the N-state E-HMM have already been estimated and calculated, the 2N-state E-HMM
The initial parameters π _2N (i) and a _2N (i) of the initial state probability and the state transition probability of the MM are calculated as shown in the following _Expressions 13 and 14, respectively.

[0032]

Π _2N (i) = 0.5 × π _N (fru (i / 2)); i = 1, 2,..., 2N

A _2N (i, j) = 0.5 × a _N (fru (i / 2),
fru (j / 2)); i = 1, 2,..., 2N, j = 1, 2,.

Here, the function fru (•) is a function for calculating an integer value obtained by rounding up the decimal part of the argument. Number 1
The physical meaning of 3 is that, for each of the two states, the initial parameter π _N (fr
By calculating two initial parameters having u (i / 2)) and multiplying each of the two initial parameters by 0.5 for normalization, the number of states is doubled and the number of states is doubled. The initial parameter π _2N (i) of the initial state probability when the number of states is 2N, which is twice, is calculated. Further, the physical meaning of the expression 14 is that the initial parameter a _N (fru (i
/ 2), fru (j / 2)), and by multiplying the four initial parameters by 0.5 for normalization, the number of states is doubled. Thus, the initial parameter a _2N (i) of the state transition probability when the number of states is 2N, which is four times as large, is calculated. Further, an initial parameter b _2N (i, j, k) of the symbol output probability is represented by the following _equation (15).

[0034]

B _2N (i, j, k) = b _N (fru (i / 2, j /
2, k)) × random (i, j, k) i = 1, 2,..., 2N, j = 1, 2,..., 2N, k = 1, 2,.

Here, the function random (•) is a function for calculating a random number generated by a predetermined conventional random number generation method using the arguments i, j, and k as initial values. The initial parameter b _2N (i, j, k) of the symbol output probability is further normalized so that the following equation 16 is satisfied. That is, normalization is performed so that the sum of the symbol output probabilities when the number of states is 2N is 1.

[0036]

(Equation 16)

Therefore, the physical meaning of Equations 15 and 16 is that, for each of the four states, the initial parameter b _N (fru (i / i /
2), four initial parameters having fru (j / 2), k) are calculated. If the same parameters are used, the Balm-Welch learning algorithm cannot be applied. Different random numbers calculated by a predetermined random number generation method.
m (i, j, k) is multiplied by the four calculated initial parameters, and further multiplied by a coefficient for normalization, whereby the number of states is doubled and the number of states is doubled. The initial parameter b _2N (i) of the symbol output probability when the number of states is 4N, which is four times, is calculated.

Hereinafter, with respect to the E-HMM having a large number of states, the same processing is repeated so that the small symbol output probability is deleted and the number of states is successively doubled. -A statistical language model of the HMM can be obtained. A specific algorithm, that is, a Balm-Welch learning algorithm used as a language model creation process for creating a statistical language model up to the state Nd will be described below with reference to the flowchart of FIG. I do.

First, in step S1, based on learning text data composed of word strings stored in the learning text data memory 10, initial parameters π ₁ (i), a ₁ (i) of the _one -state E-HMM are set. , K), b ₁ (i,
j, k) are estimated using equations (10) to (12) and stored in the model memory 12. Then, step S
In 2, based on the same learning text data,
The parameters of the one-state E-HMM are re-estimated using a Balm-Welch learning algorithm and stored in the model memory 12. At this time, when the symbol output probability is smaller than a threshold value, for example, preferably 10 ⁻³⁰⁰ , the symbol output probability is set to 0, and the state of the transition of the symbol output probability is determined from the re-estimation parameter in the model memory 12. delete. Then, in step S3,
The control parameter N indicating the number of states is set to 2.

Next, in step S4, based on the learning text data composed of the word strings stored in the learning text data memory 10, initial parameters π _N (i), a _N (i) of the _N -state E-HMM are obtained. , K), b _N (i,
j, k) are estimated using equations (10) to (12) and stored in the model memory 12. Then, step S
In 5, based on the same learning text data,
The parameters of the N-state E-HMM are re-estimated and calculated using the Balm-Welch learning algorithm and stored in the model memory 12. At this time, when the symbol output probability is smaller than a threshold value, for example, preferably 10 ⁻³⁰⁰ , the symbol output probability is set to 0, and the state of the transition of the symbol output probability is determined from the re-estimation parameter in the model memory 12. delete. Then, the process proceeds to step S6, where it is determined whether or not the control parameter N is a desired number of states Nd. If N = Nd (YES in step S6)
This language model creation processing ends. On the other hand, if N ≠ Nd (NO in step S6), the control parameter N is doubled in step S7 and set as the control parameter N, and the processing in steps S4 and S5 is repeated. That is, the number of states is doubled, and learning of re-estimation calculation is repeated.

After the above-described language model creation processing, the language model creation unit 11 stores the statistical language model after re-estimation calculation stored in the model memory 12 into the statistical language model memory 7.
Copy to and store. The configuration and operation of the speech recognition apparatus using the statistical language model of the present embodiment, which is the network grammar with probabilities using the E-HMM created and stored as described above, will be described.

In FIG. 1, a uttered voice of a speaker is input to a microphone 1 and converted into a voice signal, and then input to a feature extracting unit 2. After performing A / D conversion on the input audio signal, the feature extraction unit 2 performs, for example, LPC analysis, and obtains a 34-dimensional feature parameter including logarithmic power, 16th cepstrum coefficient, Δlog power, and 16th Δcepstrum coefficient. Is extracted. The time series of the extracted feature parameters is input to the phoneme matching unit 4 via the buffer memory 3.
HMM in HMM memory 5 connected to phoneme matching unit 4
Is composed of a plurality of states and arcs indicating transitions between the states. Each arc has a transition probability between states and an output probability for an input code. The phoneme matching unit 4 performs phoneme matching processing based on the input data, and outputs phoneme data to the speech recognition unit 6.

The statistical language model memory 7 for storing the created statistical language model is connected to the speech recognition unit 6. The speech recognition unit 6 refers to the statistical language model in the statistical language model memory 7 and determines a predetermined One pass.
Using the DP algorithm, the input phoneme data is processed from left to right without regression to determine a word having a higher probability of occurrence as speech recognition result data, thereby executing speech recognition processing and determining. Then, the obtained speech recognition result data (character string data) is output.

The present inventor performed a simulation for creating a statistical language model of a network grammar with probabilities from a large amount of word string data using the above-described algorithm for language model creation processing. The results are shown below. As the text data for learning, “dialog by telephone for inquiries about international conferences” in the dialogue database owned by the present applicant is used. Table 1 shows the simulation conditions.
Shown in

[0045]

[Table 1] Conditions for language model generation simulation ──────────────────────────────── HMM structure State transition output type ─数 Number of learning vocabulary 6420 words ──────────────数 Number of text data for learning 8475 sentences ─────────────────────────総 Total number of words 57354 40 in retraining of HMM parameters Iterations Termination condition of Balm-Welch algorithm ────────────────────────────────

First, FIG. 4 shows a change in entropy with an increase in the number of states. For comparison, FIG. 4 shows a word bigram and a word trigram.
Changes in entropy for a speech recognizer using a statistical language model including m) are also shown. It can be seen from FIG. 4 that the entropy of the apparatus of this embodiment is lower than that of a word bigram when the number of states is 128 or more. In addition,
In this embodiment when the number of states is 512, the number of parameters having a value in the symbol output probability is 2106967. Therefore, the language model creation processing of the present embodiment requires only 0.125% of the memory amount of the model memory 12 and the calculation amount of the language model creation unit 11 when compared with the case where only the basic Balm-Welch learning algorithm is used. Has been reduced.

Next, the present inventor uses the statistical language model to learn the E-HMM learned by the language model creating section 11 of the present embodiment, and uses the speech recognition apparatus of FIG. A simulation was performed. The results are described below. In the continuous speech recognition simulation, a continuous distribution HMM was used as a phoneme model, a beam search was used as a search algorithm, and an E-HMM was used as a language model. Test data of the uttered voice (hereinafter referred to as test data) includes a memory 1 used for E-HMM learning.
38 sentences of the same task as the learning text data in 0 were used. The evaluation was performed at the sentence recognition rate. Also, a simulation was performed when a statistical language model was created by adding the test data to the learning text data. Other simulation conditions are shown in Table 2 below.

[0048]

[Table 2] Conditions for continuous speech recognition simulation ──────────────────────────── Phoneme acoustic model 4-state 3-loop mixed distribution HMM ─ ─────────────────────────── Acoustic parameter log power + 16th order LP cepstrum + Δlog power + 16th order Δ cepstrum ─────────男性 One male announcer for learning, uttering 2620 words Text data ───────────────────数 Number of recognized words 435 words ──────────────────────────── Beam width 4096 ───── ─────────────────────── Test data Same speaker's utterance 38 sentences ───────── ────────────────── speaking style recitation ────────────────────────────

FIG. 5 shows a simulation result when a statistical language model is created by adding test data to learning text data. Also, for comparison, the results of simulation when speech recognition is performed using a statistical language model including a word bigram and a word trigram are shown. As can be seen from FIG. 5, the number of states is 128.
, A sentence recognition rate higher than that of a word bigram was obtained.

FIG. 6 shows the results of a simulation when a statistical language model was created using only the learning text data. FIG. 6 shows the word percent correct and the word percent accuracy together with the sentence recognition rate. The results of a simulation of a statistical language model including a trigram of words are also shown for comparison. Here, the percent correct of the word and the percent accuracy of the word are expressed by Expression 17 and Expression 18, respectively, as already determined in the technical field.

[0051]

## EQU17 ## Percent Collect = {(N−D−S) / N} ×
100 [%]

## EQU18 ## Percent accuracy = {(NDS-
I) / N} × 100 [%]

Here, N is the number of all words, D is the number of missing errors, S is the number of replacement errors, and I is the number of insertion errors. As is clear from FIG. 6, in this simulation, a sentence recognition rate of 44.7% was obtained in the E-HMM having 128 states. On the other hand, the sentence recognition rate in the case of the word trigram model was 36.8%. Therefore, the sentence recognition rate could be improved, and the results of these simulations showed the effectiveness of the language model creation processing of the present embodiment.

In the above embodiment, the feature extracting unit 2
Phoneme matching unit 4, speech recognition unit 6, language model creation unit 1
1 is constituted by, for example, a digital computer.

As described above, by using the language model creation processing of this embodiment, the amount of memory of the model memory 12 and the amount of calculation of the language model creation unit 11 can be greatly reduced. -A statistical language model including an HMM can be created by learning using a Balm-Welch learning algorithm. When the created statistical language model is applied to a speech recognition device, a higher speech recognition rate can be obtained as compared with the conventional example.

The Balm-Welch learning algorithm used in the present embodiment locally learns to minimize the error in the probability of each state transition, so the initial parameters are important. In this embodiment, when the number of states is sequentially increased, the symbol output probability is weighted by a random number. , A statistical language model that is a network grammar with probability may be created. When speech recognition is performed using a network grammar with probabilities using this thesaurus, a higher speech recognition rate can be obtained than when speech recognition is performed using the statistical language model of the present embodiment. it is conceivable that.

In the above embodiment, the primary E-HM
M was used, but the present invention is not limited to this, and the secondary E-HM
The statistical language model may be created by applying the language model creation processing of the present embodiment using M in the same manner. In the above embodiment, the language model creation processing using the Balm-Welch learning algorithm, in which the amount of memory in the model memory 12 and the amount of calculation in the language model creation unit 11 are reduced, is performed using a statistical language Although used for creating a model, the present invention is not limited to this, and may be applied to a morphological analyzer or the like.

[0057]

As described above in detail, according to the present invention, the initial state probability and the state transition probability are determined based on the storage device for storing the statistical language model during learning and the input learning text data. Control means for learning and creating a statistical language model for speech recognition or morphological analysis, which is an ergodic hidden Markov model having a predetermined number of states including a parameter of a symbol output probability, using a Balm-Welch learning algorithm In the statistical language model creating apparatus having the above, when the control means estimates and calculates the parameters using the Balm-Welch learning algorithm, when the symbol output probability is smaller than a predetermined threshold value , The symbol output probability is set to 0,
Deleting means for deleting the state of the transition of the set symbol output probability from the storage device; and, when learning the parameters using the Balm-Welch learning algorithm, the number of states of the Ergodic hidden Markov model is Parameter calculation means for calculating the parameters of the Ergodic Hidden Markov Model when the number of states is set to be twice as large and the number of states is larger. Therefore, while the amount of memory of the storage device and the amount of calculation of the control unit can be significantly reduced by the reduction unit, an E-HMM having a larger number of states can be created by the parameter calculation. That is, a statistical language model including an E-HMM can be created by learning using a Balm-Welch learning algorithm. The created statistical language model can be applied to a speech recognition device or a morphological analysis device. At this time, a higher speech recognition rate or a lower morphological analysis error can be obtained as compared with the conventional example. .

[Brief description of the drawings]

FIG. 1 is a block diagram of a speech recognition apparatus according to an embodiment of the present invention.

FIG. 2 is a state transition diagram showing an example of an ergodic HMM used in the speech recognition device of FIG.

FIG. 3 is a flowchart illustrating a language model creation process executed by a language model creation unit in FIG. 1;

FIG. 4 is a graph showing entropy with respect to the number of states as a result of the simulation of the speech recognition device of FIG. 1;

5 is a graph showing a sentence recognition rate with respect to the number of states as a result of simulation of the speech recognition apparatus of FIG. 1 when a statistical language model is created by adding test data to learning text data.

6 is a graph showing a recognition rate with respect to the number of states as a result of simulation of the speech recognition apparatus in FIG. 1 when a statistical language model is created based on only learning text data.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... microphone, 2 ... feature extraction part, 3 ... buffer memory, 4 ... phoneme collation part, 5 ... One pass DP speech recognition part, 6 ... Hidden Markov model (HMM) memory, 7 ... statistical language model memory Learning text data memory, 11: Language model creation unit, 12: Model memory.

Continuation of the front page (56) References JP-A-6-202687 (JP, A) Proceedings of the Acoustical Society of Japan (Oct.-Nov. 1994) 1-Q-6, P.S. 153-154 IEICE Technical Report "Sound" SP93-26, P.E. 17-24, June 1993, PROCEEDINGS OF THE IEEE INTERNATIONAL AL CONFERENCE ON A COUSTICS, SPEECH AND SIGNAL PROCESSING G, P.E. I-357 to 360. 1994 (1994)

Claims (3)

(57) [Claims] 1. A storage device for storing a statistical language model during learning, and a predetermined number of states including parameters of an initial state probability, a state transition probability, and a symbol output probability, based on input learning text data. An ergodic hidden Markov model, a statistical language model for speech recognition or morphological analysis, learning means using a learning algorithm of Balm-Welch, and a statistical language model creating apparatus comprising: The control means, when estimating and calculating the parameters using the Balm-Welch learning algorithm, when the symbol output probability is smaller than a predetermined threshold, the symbol output probability is set to 0, Deleting means for deleting, from the storage device, a state at the transition of the set symbol output probability, the balm-welch When learning the above parameters using the learning algorithm described above, the number of states of the ergodic hidden Markov model is sequentially set to twice, and the above parameters of the ergodic hidden Markov model when the number of states is larger are calculated. A statistical language model creation device, comprising: a parameter calculation unit. 2. The method according to claim 1, wherein the parameter calculation means calculates a second state number parameter which is twice the first state number from the first state number. By calculating two parameters having the same initial parameters of the same initial state probability at the number of states as, and multiplying the calculated two initial parameters by 0.5 for normalization, the second state First calculating means for calculating initial parameters of initial state probabilities at the time of numbers; calculating four parameters having the same initial parameters of state transition probabilities at the first number of states for each of the four states A second calculating means for calculating the initial parameters of the state transition probability at the second number of states by multiplying the calculated four initial parameters by 0.5 for normalization; For each state, when the first number of states Are calculated, and the calculated four initial parameters are respectively multiplied by different random numbers generated by a predetermined method, and the multiplication result is referred to as a second state. A third calculating means for calculating a parameter of the symbol output probability at the time of the second number of states by normalizing so that the sum of the symbol output probabilities at the time of the number is one. The statistical language model creating apparatus according to claim 1, wherein 3. A statistical language model creating apparatus according to claim 1, wherein said statistical language model is created for speech recognition. 3. The statistical language model creating apparatus according to claim 1, further comprising: And a speech recognition unit for recognizing the uttered speech by referring to a statistical language model created by the statistical language model creation device.