JP2002229588A - Statistical language model forming system, speech recognizer and statistical language model forming method as well as recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problems that, if a statistical language model is formed from a language corpus 201 having an extremely high sparse characteristic, there is a tendency to inclusion of words having a high unigram frequency as the recognition result of speech using the same and a recognition error is liable to occur. SOLUTION: This system is provided with word connection probability calculating means of inputting a backoff probability to a monotonously increasing function and determining the result of the computation by this function as a fresh conditional probability of a word chain in making calculation by subjecting the conditional probability of the word chain having a low appearance frequency to backoff smoothing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a statistical language model creating apparatus for creating a statistical language model used for speech recognition, and more particularly to a low-dimensional word by reducing a dynamic range of a frequency difference when backing off. (Chain) Statistical language model generation device capable of preventing a word string including a word having a high frequency from having an unduly high language probability, a speech recognition device using the same, a statistical language model creation method, and a recording medium It is about.

[0002]

2. Description of the Related Art A statistical language model generally describes a restriction on a chain of words, and is used as information that gives information about a word string that is likely to be continued in continuous speech recognition.

FIG. 13 is a block diagram showing the configuration of a conventional speech recognition apparatus for recognizing continuous speech disclosed in Japanese Patent Application Laid-Open No. 2000-99086. In the figure, reference numeral 101 denotes an acoustic model for performing pattern matching on an input speech in units of phonemes, syllables, or words. A recognition candidate that is acoustically similar is obtained by pattern matching. In the illustrated example, the acoustic model 1 stored for each word
01 is used for pattern matching. 102
Is an acoustic probability calculation unit that performs pattern matching of the input voice using the acoustic model 101. In the illustrated example, the probability that the input voice occurs from a certain word is calculated for all the words using the acoustic model 101 for each word. I do. 10
3 describes constraints on a chain of words (statistical)
A language model 104 is a language probability calculation means.
The probability of occurrence of the word whose probability has been calculated is calculated, and the word having the largest combined probability of the obtained probability and the probability calculated by the acoustic probability calculation means 102 is output as a recognition result.

Next, the operation will be described. First, the acoustic probability calculation unit 102 performs pattern matching on the input speech for each word (string) using the acoustic model 101 stored in the storage medium in advance. At this time, assuming that the input speech is X and a certain word (string) is w _k , the probability P (X | w _k ) of the input speech X occurring from a certain word (string) w _k is calculated by the acoustic probability calculation means 102. Calculated for words (columns).

[0005] Thereafter, the language probability calculating means 104 stores the language model 103 stored in a storage medium (not shown) in advance.
, The language probability P (w _k ) of the occurrence of a word (column) w _k
Is calculated for all words (columns) w _k . Then, P (X | w _k ) P (w _k ) is calculated from the language probability and the probability calculated by the acoustic probability calculation means 102, and the word (string) having the largest value is output as a recognition result.

FIG. 14 is an explanatory diagram for explaining generation of a statistical language model used in the speech recognition apparatus of FIG. In the figure, reference numeral 201 denotes a language corpus used for constructing a language model, which means a language database composed of many sentences such as newspapers. 202 is an N-gram estimating means, which is a language model corpus 201 or 00
（(N-1) Gram probability model is used to perform smoothing by manipulating probability values in a direction in which the conditional probabilities of all word chains are equal in the N-gram conditional probabilities.
It outputs a Gram language model 203. An N-gram language model 203 generally used as the language model 103 is a model that describes the probability of occurrence of a chain of N words. This N-gram language model 203 obtains the conditional probability of an N-gram corresponding to the probability that the next word will come when the immediately preceding word string is (N-1) arbitrary words. The attached probabilities are held for word strings of all N words that can appear. 204-0
-204- (N-1) are storage means, which are used during the N-gram estimation by the N-gram estimation means 202;
(N-1) The probability model of the gram is stored. Where 0
Assuming that the gram (zero chain) probability is, for example, the reciprocal of the maximum number of word types is the zero gram probability of each word, all words have the same value.

First, an outline of generation of a language model will be described. If a large value is taken as N of the N-gram language model 203, the number of combinations becomes enormous.
Are used as values such as 2 and 3. A case where N = 2 is called a bigram (two chains), and a case where N = 3 is a trigram (three chains). Hereinafter, a trigram will be described as an example. The conditional probability in this trigram is
For example, if the immediately preceding word string (the two preceding words (strings)) is w
_1, when he was _{w 2,} the words that come to this after the conditional probability _P is _{w 3 |} represented by _{(w 3 w 1, w 2} ). This conditional probability is calculated for all possible word strings (w ₁ , w ₂ ,
What is held (or made computable) for w ₃ ) is the trigram language model.

[0008] The conditional probability is generally obtained by counting the events actually occurring in a certain large database. For example, in a trigram, the conditional probability P (w ₃ | w ₁ , w ₂ ) is equal to the count number N (w ₁ , w ₁ , w ₁ , w ₃ ) of the word string (w ₁ , w ₂ , w ₃ ).
w ₂ , w ₃ ) by the count N (w ₁ , w ₂ ) in which the word string (w ₁ , w ₂ ) appears. However, in such an N-gram model, the number of combinations is enormous in practical use, and there is a problem that N-grams that do not appear in the database even once appear or that there are many N-grams that appear only a few times. This defect is called sparsity. At this time, the appearance probability of a word string corresponding to an N-gram that does not appear in the database at least once has a value of 0, and the N-gram that appears only a few times has a very small probability, the estimation accuracy is low, and the recognition performance is low. It causes deterioration.

Therefore, in order to solve the above problem, N
The gram estimating means 202 performs smoothing for manipulating the value of the conditional probability of the N-gram. The smoothing of the conditional probability of the N-gram is often performed using the conditional probability of the (N-1) -gram. For example, the conditional probability that the next word _{w 3} of the word chain _w 1 -w ₂ appears, that is, conditional probability of the tri-gram P _(w 3 |
The smoothing of w ₁ , w ₂ ) is performed using the bigram conditional probability P (w ₃ | w ₂ ).

Next, the operation of generating a language model will be described. First, 0-gram probability is set in the N-gram estimating means 202, and this is stored as a 0-gram probability model in the storage means 204.
Store to −0. Next, smoothing of n (0 ≦ n ≦ N) grams by the N-gram estimating means 202 is sequentially performed from n = 1 to N. The storage means 204-
(N-1) gram stored in (n-1) ((n-1)
The word sequence of the chain is used, and during n <N, the estimated n-gram conditional probability is stored in the storage means 204-n as an n-gram probability model. Thereafter, when n = N, the storage means 204-0 to 204- (N-1)
Are sequentially output as the N-gram language model 203.

A technique called back-off is generally used for the smoothing operation by the N-gram estimating means 202 as described above. This back-off is described in “Speech Information Processing” (Kenji Kita, Satoshi Nakamura, Masaaki Nagata, Morikita Publishing Co., Ltd., 1996), p. 34, the sum of the probability estimates of the word chains existing in the language corpus 201 is estimated to be smaller than 1, and the rest is estimated to be the language corpus 2
This is an approximation method that assigns to the chain probabilities of word chains that are not 01 (the number of N-grams with a small number of observations is further reduced, and the sum of the surplus probability values is added to the N-grams for which no observations were made. Distribute based on value).

When the word chain w ₁ -w ₂ -w ₃ does not exist in the language corpus 201 (the frequency N (w ₁ , w ₂ , w ₃ ) of the word chain in the language corpus 201) = 0, w _1-
The estimated probability P (w ₃ | w ₁ , w ₂ −w ₃₎ of the word chain
w ₂ ) is represented by the following equation (1). P (w ₃ | w ₁ , w ₂ ) ∝P (w ₃ | w ₂ ) (1) Furthermore, when the word chain of the word chain w ₂ −w ₃ is not in the language corpus 201 (in the language corpus 201). The word chain frequency N (w ₂ , w ₃ ) = 0) and the estimated probability P (w ₃ | w ₂ ) of the word chain w ₂ −w ₃ are expressed by the following equation (2). P (w ₃ | w ₂ ) ∝P (w ₃ ) (2)

As a more specific expression of the above equations (1) and (2), a typical back-off method will be described.
For example, Clarkson et al. "Stati
physical Language Modeling
Using The CMU-CAMBRIDGE T
OOOLKIT, "EuroSpeech '97, Vo
l. 5, pp. The Witten-Bell method cited in 2707-2710 (1998) is well known. The Witten-Bell method uses a probability estimate P
(W ₃ | w ₁ , w ₂ ) and P (w ₂ | w ₁ ) are represented by the following equations (3) and (4).

(Equation 1) Here, α (w ₁ ) and β (w ₁ , w ₂ ) are probability normalization constants, and R (w ₁ , w ₂ ) is w ₁ in the language corpus 201.
the type number of words that appear in the next w _2, R (w ₁₎ is the number of types of words that appear in the following w ₁ in corpus within 201. N (w ₁ , w ₂ , w ₃ ) is the language corpus 2
The number of occurrences of the word chain w ₁ w ₂ w ₃ in 01, N
(W ₁ , w ₂ ) is a word chain w in the language corpus 201
N (w ₁ ) is the number of occurrences of ₁ w ₂ and N (w ₁ ) is the language corpus 2
Word w ₁ is shows the number of times that appear in the 01. P
(W ₁ ) is generally P (w ₁ ) = N (w ₁ ) / N (*)
In are those represented, this is referred unigram probability of the word w _1. In addition, * represents all the words in the language corpus 201.

[0014] Further, in order to eliminate the sparseness described above,
Another approach to this is "speech information processing" (Kitaken
2, Satoshi Nakamura, Masaaki Nagata, Morikita Publishing Co., Ltd., 199
6 years) p. The class language model described in 35
There is something called. It categorizes words into classes and
Is used to estimate the chain probability between word classes.
It is intended to be used for fixed values. Where the word
w₁, W₂, ..., each of which is w_i∈C_iThat
Assuming that it belongs to the class, the word string w₁,
w ₂, ..., w_i-1After the appearance of the word w_iAppears
Is approximated as in the following equations (5) to (7)
Is done. P (w_i| W₁, W₂, W₃, ..., w_i-1) To P (w_i｜ C_i) ・ P (C_i｜ C₁, C₂, ..., C_i-1) (5) Especially in the case of the bigram model, the following equation (6) is used.
become. P (w_i| W₁, W₂, W₃, ..., w_i-1) To P (w_i｜ C_i) ・ P (C_i｜ C_i-1(6) Further, in the case of the trigram model, the following equation (7) is used.
become that way. P (w_i| W₁, W₂, W₃, ..., w_i-1) To P (w_i｜ C_i) ・ P (C_i｜ C_i-2, C_i-1) ・ ・ ・ (7)

Each conditional probability is generally determined from the frequency of word chains in the language corpus 201 as shown in the following equation (8). _{P (w i | C i)} = N (w i) / N (∀w∈C i) P (C i | C i-2, C i-1) = N (C i-2, C i-1 _{, C i) / N (C} i-2, C i-1, *) ··· (8) here, N _{(w i)} the frequency with which the word _{w i} appears in the language corpus 201, N ( w, x) is the frequency at which the word chains w, x appear in the language corpus 201, and N (C,
D) indicates the frequency at which words belonging to class C and words belonging to class D are linked.

[0016]

Since the conventional statistical language model generating apparatus is configured as described above, the statistical linguistic corpus 201 having a very high sparseness (that is, very often back-off) is used. When a statistical language model is generated, there is a problem that a word having a high unigram frequency is likely to be included as a result of speech recognition using the model, and a recognition error is likely to occur.

The above problem will be specifically described. In the case of the method using back-off smoothing, for example, when the vocabulary includes “Japan”, “no (case particle)”, “he (case particle)”, and “picture”, and the utterance is “Nihononoe” When calculating probabilities, all of these word chains are
If it is not 1, the back-off results in the following equation (9). P ("picture" | "Japan", "") ∝P ("picture") P ("he" | "Japan", "") ∝P ("he") ... (9) where Since the case particle "he" generally has a much higher frequency than "picture", the following expression (10) is obtained. P ("picture" | "Japan", "") ≪P ("he" | "Japan", "of") ... (10) As can be seen from equation (10), the language probability P ("picture" ”|”
There is a large difference between "" of "Japan" and "" of "" and ") of"", so that the sparseness is very high (that is, the case of back-off is very low). Many)
In a language model created from the language corpus 201, words having a high unigram frequency are likely to be included as recognition results.
There is a problem that recognition errors easily occur.

In addition to the above, when a class language model is used, there is a problem that the accuracy of the word chain probability of the statistical language model is reduced.

The above problem will be specifically described. In the class language model, generally, the chain probability estimation value P (w ₃ | w ₁ , w ₂ ) of the word chain w ₁ −w ₂ −w ₃ is expressed by the following equation (1).
Given in 1). P (w ₃ | w ₁ , w ₂ ) = P (C ₃ | C ₁ , C ₂ ) · P (w ₃ | C ₃ ) (11) where C ₁ , C ₂ , and C ₃ are The classes to which w ₁ , w ₂ , and w ₃ belong respectively are shown. At this time, w ₃ ∈C
_If the word “ ₃ ” is given, the conditional probability that the word w ₃ ′ is chained next to the words w ₁ and w ₂ is expressed by the following equation (12).
Given by P (w ₃ ′ | w ₁ , w ₂ ) = P (C ₃ │C ₁ , C ₂ ) · P (w ₃ ′ | C ₃ ) (12) where P (w ₃ │w ₁ , W ₂ ) and P (w ₃ ′ | w ₁ ,
w ₂ ) are P (w ₃ | C ₃ ) and P (w ₃ ′ | C
₃ ) Big and small. P (w ₃ | C ₃ ) is generally given by the following equation (13) as described above. P (w ₃ | C ₃ ) = N (w ₃ ) / N (C ₃ ) (13) From equation (13), P (w ₃ | w ₁ , w ₂ ) and P (w ₃ ′)
The magnitude of | w ₁ , w ₂ ) ultimately depends on the magnitude of the appearance frequency (unigram) in the class.

In such a case, the word chain frequency N (w ₁ w
_{Even if 2} w ₃ ) ≫N (w ₁ w ₂ w ₃ ′), the occurrence frequency in the class is N (w ₃ ) / N (C ₃ ) <N (w ₃ ′) / N
In the case of (C ₃ ), the relationship becomes as shown in the following expression (14). P (w ₃ | w ₁ , w ₂ ) <P (w ₃ ′ | w ₁ , w ₂ ) (14) Therefore, the magnitude relation of the probabilities is not correctly expressed, and the accuracy of the word chain probability deteriorates. Resulting in.

The present invention has been made in order to solve the above-mentioned problem, and alleviates the dynamic range of the frequency difference in the case of back-off so that the word (chain) frequency of a low dimension such as the unigram frequency is large. An object of the present invention is to provide a statistical language model generation device and method capable of preventing a word string including a word from having an unduly high language probability.

It is another object of the present invention to provide a statistical language model generating apparatus and method for preventing a decrease in word chain probability accuracy due to the use of a class language model and providing a high accuracy language model.

A further object of the present invention is to obtain a speech recognition device with improved speech recognition accuracy using the statistical language model generated by the statistical language model generation device.

Still another object of the present invention is to provide a computer-readable recording medium on which a program for causing a computer to implement the above-described statistical language model generation method is recorded.

[0025]

A statistical language model generating apparatus according to the present invention includes a corpus storing means for storing a language corpus, and the appearance frequency and the number of word types of word chains existing in the language corpus. Word chain frequency counting means for counting the frequency of word chains, word connection probability calculating means for calculating the conditional probability of word chains based on the frequency of the word chains counted by the word chain frequency counting means,
When the word connection probability calculation means calculates the conditional probability of a word chain having a low frequency of occurrence by performing back-off smoothing, the function is input to a function that monotonically increases the back-off probability. And a word connection probability recalculating means for obtaining a conditional probability.

In the statistical language model generating apparatus according to the present invention, a monotonically increasing function is a power function.

In the statistical language model generating apparatus according to the present invention, a function that monotonically increases is a sigmoid function.

In the statistical language model generating apparatus according to the present invention, the function that monotonically increases has upper and lower limits in the operation result.

A statistical language model generating apparatus according to the present invention includes a corpus dividing means for dividing a language corpus into a plurality of words, and a plurality of divided language corpuses for counting the frequency of word chains by the word chain frequency counting means. Parameter setting means for classifying into a language corpus and a language corpus for language performance evaluation, and setting a parameter of a monotonically increasing function used by the word connection probability recalculation means, and words counted in the language corpus for frequency counting The conditional probability of the word chain in the language corpus for language performance evaluation is calculated based on the frequency of the chain, and the language performance is evaluated by using the conditional probability. Best linguistics every time the language corpus is reclassified and parameters of monotonically increasing functions are changed Detecting a parameter that gives the,
Optimum parameter determining means for determining an average value of these parameters as an optimum parameter.

In the statistical language model generating apparatus according to the present invention, the plurality of language corpuses divided by the parameter setting means are classified into a language corpus for counting frequencies related to word chains and a language corpus for evaluating language performance. In addition, these are made into a plurality of kinds of combinations, different parameters are set for each combination, and the optimum parameter determining means detects a parameter that gives the highest linguistic performance in each combination, and determines an average value of these parameters as an optimum parameter. Is what you do.

A statistical language model generating apparatus according to the present invention includes a corpus storing means for storing a language corpus, and a frequency associated with a word chain including the frequency of occurrence of word chains and the number of types of words existing in the language corpus. Word chain frequency counting means for counting, word connection probability calculating means for calculating conditional probability of word chains based on the frequency of word chains counted by the word chain frequency counting means, and word chains existing in the language corpus And a similar word chain adding means for newly adding a word chain similar to to the language corpus.

A statistical language model generating apparatus according to the present invention includes word class storage means for storing class information obtained by classifying words existing in a language corpus into word classes having similar meanings, and a similar word chain adding means. Determines whether a word belonging to a predetermined word class is included in a word chain in a language corpus based on the class information, and replaces a word belonging to the predetermined word class with another word belonging to this word class. The word chain replaced with the word is newly added to the language corpus as a similar word chain.

The statistical language model generating apparatus according to the present invention includes a corpus storing means for storing a language corpus, and a frequency associated with a word chain including the frequency of occurrence of a word chain and the number of word types existing in the language corpus. Word chain frequency counting means for counting, word class storage means for storing class information obtained by classifying words existing in the language corpus into word classes having similar meanings, words existing in the language corpus based on the class information A word-class frequency counting means for counting the frequency of occurrence of the chain for each word class to which the words included in the word chain belong; and calculating a conditional probability of the word chain based on the frequency related to the word chain, Based on the appearance frequency of the word chain calculated by the frequency counting means, the conditional probability of the word class to which the word included in the word chain belongs To calculate the one in which these and a word connection probability calculation means for a new conditional probability of a word chain probabilities obtained by linear combination.

A speech recognition apparatus according to the present invention recognizes input speech using a statistical language model obtained by the above-described statistical language model generation apparatus.

The statistical language model generating method according to the present invention includes a word chain frequency counting step of counting the frequency of a word chain including the number of appearances of word chains and the number of word types existing in a language corpus; A word connection probability calculating step of calculating a conditional probability of a word chain based on the frequency relating to the word chain counted in the word chain frequency counting step, and a word connection probability calculating step of calculating a conditional probability of a word chain having a low appearance frequency In performing the back-off smoothing in the calculation, the back-off probability is input to a function that monotonically increases, and the result of the operation by this function is used as a new conditional probability of a word chain. A statistical language model is generated using the conditional probabilities of word chains obtained in the connection probability calculation step and the word connection probability recalculation step. It is intended and a statistical language model generating step of.

In the statistical language model generating method according to the present invention, a monotonically increasing function is a power function.

In the statistical language model generation method according to the present invention, a monotonically increasing function is used as a sigmoid function.

In the statistical language model generation method according to the present invention, the function that monotonically increases has upper and lower limits in the operation result.

In the statistical language model generation method according to the present invention, a language corpus is divided into a plurality of languages, and the language corpus is classified into a language corpus for counting frequencies and a language corpus for evaluating language performance, and word connection is performed. A parameter setting step of setting a parameter of a monotonically increasing function used in the probability recalculation step, and a word chain in a language corpus for language performance evaluation based on the frequency of the word chain counted in the language corpus for frequency counting Calculate the conditional probability of, the language performance evaluation step to evaluate the language performance using this conditional probability, and obtain a parameter that gives the best language performance every time the parameter setting step and the language performance evaluation step are repeated, An optimal parameter determining step of determining an average value of these parameters as an optimal parameter. Is shall.

In the statistical language model generation method according to the present invention, in the parameter setting step, the plurality of divided language corpuses are classified into a language corpus for counting the frequency of word chains and a language corpus for evaluating language performance. Then, these are made into a plurality of kinds of combinations, different parameters are set for each combination, and in the optimum parameter determination step, the parameter that gives the best language performance in each combination is detected, and the average value of these parameters is calculated. This is determined as the final optimum parameter.

The statistical language model generation method according to the present invention includes a similar word chain adding step of newly adding a word chain similar to a word chain existing in a language corpus to a language corpus, and a word existing in the language corpus. A word chain frequency counting step of counting the frequency of the word chain including the frequency of occurrence of the chain and the number of word types; and a condition of the word chain based on the frequency of the word chain counted in the word chain frequency counting step. A word connection probability calculation step of calculating a probability; and a statistical language model generation step of generating a statistical language model using the conditional probability of the word chain obtained in the word connection probability calculation step. .

In the statistical language model generating method according to the present invention, in the similar word chain adding step, words belonging to a predetermined word class are classified into words in the language corpus based on class information obtained by classifying words existing in the language corpus. It is determined whether or not a word chain is included in a word chain, and a word chain obtained by replacing a word belonging to a predetermined word class with another word belonging to this word class is newly added to the language corpus as a similar word chain. is there.

The statistical language model generation method according to the present invention includes a word chain frequency counting step of counting the frequency of a word chain including the number of occurrences of word chains and the number of word types existing in a language corpus; Based on class information obtained by classifying words existing in the corpus into word classes having similar meanings, the frequency of occurrence of word chains existing in the language corpus is counted for each word class to which the words included in this word chain belong. Calculating the conditional probability of the word chain based on the word class frequency counting step and the word chain frequency, and including the word chain occurrence frequency counted in the word class frequency counting step in the word chain. Calculate the conditional probabilities of the word class to which the word belongs, and calculate the probability obtained by linearly combining them with a new conditional And word connection probability calculation step of the rate, in which and a statistical language model generating step of generating a statistical language model using the conditional probability of a word chain obtained by this word connection probability calculation step.

A recording medium according to the present invention stores a program for causing a computer to execute the above-described statistical language model generation method.

[0045]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. Embodiment 1 FIG. FIG. 1 is a diagram showing a configuration of a statistical language model generation device according to Embodiment 1 of the present invention. In FIG. 1, reference numeral 1 denotes a corpus storage unit for storing a language corpus used for constructing a language model, which is realized by a hard disk or a large-capacity storage medium of a computer system. As a language corpus, there is a language database composed of many sentences such as newspapers, for example, as described in the related art. Numeral 2 denotes a corpus input unit which reads out a desired language corpus from the corpus storage unit 1 and holds it so that the word chain frequency counting unit 3 can use it. Reference numeral 3 denotes a word chain frequency counting unit that counts the frequency related to a word chain existing in the language corpus held in the corpus input unit 2. The frequency relating to the word chain counted by the word chain frequency counting means 3 includes the frequency of appearance of the word chain existing in the language corpus, the frequency of the word type following the word chain, and the like. Reference numeral 4 denotes a word connection probability calculating unit that calculates a conditional probability that words are linked based on the word chain frequency counted by the word chain frequency counting unit 3. In the illustrated example, the Witten-Bell described in the related art is used. The conditional probability is calculated by performing smoothing by the method.

Reference numeral 5 denotes a word connection probability recalculating means, which performs back-off smoothing to calculate conditional probabilities relating to word chains whose appearance frequency is counted as 0 or as close to 0 as possible. The off-probability is input to a monotonically increasing function, and the result of operation by this function is recalculated as a new conditional probability. Reference numeral 6 denotes a language model generation unit that holds a conditional probability of each word chain calculated by the word connection probability calculation unit 4 and the word connection probability recalculation unit 5 and generates a statistical language model.

Next, the operation will be described. FIG. 2 is a flow chart showing a statistical language model generation operation by the statistical language model generation device of FIG. 1. An example in which a trigram language model is created along this flowchart will be described.
First, the corpus input means 2 reads the language corpus from the corpus storage means 1, and holds the word chain frequency counting means 3 in a usable state (step ST1).

Next, the word chain frequency counting means 3 counts the frequency of word chains from the language corpus read into the corpus input means 2 (step ST2, word chain frequency counting step). Specifically, it is assumed that vocabulary words “tomorrow”, “no”, “weather”, “train”, and “situation” are registered in the language corpus. At this time, in the language corpus, "... tomorrow's weather ..."
Times, "... tomorrow's train ..." three times, "
... the situation of tomorrow ... "
The appearance frequency of the word chain consisting of the words w ₁ , w ₂ , w ₃ is N
Expressed as (w ₁ , w ₂ , w ₃ ), the word chain frequency counting means 3 counts the appearance frequency of the word chain composed of the above words as in the following equation (15). N ("Tomorrow", "no", "weather") = 5 N ("tomorrow", "no", "train") = 3 N ("tomorrow", "no", "situation") = 4・ (15) At this time, N (“tomorrow”, “no”) = (5 + 3 + 4) =
It becomes 12. Similarly, the word chain frequency counting means 3
w _1, the frequency of occurrence of the two sets of words chain of _{w 2} N _(w 1, _w
2) and words _{w 1} of the word appearance frequency N _{(w 1)} for counting.

Further, the word chain frequency counting means 3 calculates the word type R (w ₁ , w ₁ , w ₁ , w ₂₎ following the predetermined word chain w ₁ w ₂ .
w ₂ ) is counted. In the case described above, the types of words following “tomorrow”-“no” are “weather”, “train”, and “situation”.
R (“tomorrow”, “of”) = 3. Also,
Similarly, R (w ₁ ) is counted. In the above case, the words following "tomorrow" are three "no", and R ("tomorrow") = 1 and R ("of") = 3.

Next, a word connection probability calculating means 4 considers a set of three word chains w ₁ w ₂ w ₃ and the word chain probability P
Estimate (w ₃ | w ₁ , w ₂ ). Specifically, when the given word chain w ₁ w ₂ w ₃ exists in the language corpus, that is, when N (w ₁ , w ₂ , w ₃ )> 0, the word connection probability calculation means 4 Estimates the probability according to the first equation of the following equation (16). This is Wit described in the background art.
This is the same as the smoothing of the ten-Bell method (step ST3, word connection probability calculation step).

If N (w ₁ , w ₂ , w ₃ ) = 0 and N
If (w ₁ , w ₂ )> 0, the word connection probability recalculating means 5
Backoff probability _{P (w} 3 | _{w 2)} conditional probability P represented by the following formula (17) using a _'(w 3 _| w 2)
Is calculated, and the conditional probability of the word chain whose appearance frequency is 0 is estimated according to the second expression of the following expression (16) (step ST4, word connection probability calculation step, word connection probability recalculation step).

Further, N (w₁, W₂, W₃) = 0,
N (w₁, W₂) = 0, word connection probability recalculation means
5 is the back-off probability P (w₃| W₂) Using the following formula
(17) The conditional probability P '(w₃| W₂)
, And further, according to the third equation of the following equation (16),
Estimate the conditional probability of a word chain with zero occurrence frequency
(Step ST5, word connection probability calculation step, word connection probability
Continuation probability recalculation step). At this time, the word connection probability
The calculating means 5 calculates the back-off probability P (w₂), The following equation (1
8) and conditional expression (17) using equation (19)
Probability P '(w₃| W ₂) Is calculated.

(Equation 2) Here, in Expressions (16) to (19), α
(W ₁ ) and β (w ₁ , w ₂ ) are normalization constants, δ and λ are power parameters, and δ ≦ 1, λ ≦ 1. The denominator between P ′ (w ₃ | w ₂ ) and P ′ (w ₂ ) is a normalization constant. The denominator of the right side of equation _{(17), N (w 2, w 3} ) in the corpus> 0. The take for all _{w 3.} Further, the denominator of equation (19) the right-hand side takes for all words _{w 2} as the N _(w 2)> 0 in the corpus.

Word connection probability calculating means 4 and word connection probability recalculating means 5 controlled by language model generating means 6
There, all of that can appear word chain the above processing (w _1, w
₂ , w ₃ ). As a result, the trigram language model is created with the conditional probability of each word chain held in the language model generation means 6 (step ST).
5. Statistical language model generation step).

As described above, when backing off in the calculation of the conditional probability of a word chain, the dynamic range of the frequency difference can be reduced by raising the conditional probability of a low-order word chain to the power. For example, if the ratio of the unigram probability of a certain word w ₁ and w ₂ was two-fold,
P (w ₂ ) = 2P (w ₁ ), and when it is raised to the power of 0.5, P ′ (w ₂ ) = P (w ₂ ) ^0.5 = (2
P (w ₁ )) ^0.5 = 2 ^0.5 · P (w ₁ ) ^0.5 =
The ratio becomes 1.4 times at 1.4 · P ′ (w ₁ ), and the dynamic range becomes smaller. Therefore, it is possible to prevent a word string including a word having a high unigram frequency from having an unduly high language probability.

Here, it is apparent that a similar effect can be obtained if a monotonically increasing function that relaxes the dynamic range is used instead of raising the conditional probability of a low-order word chain when backing off. is there. 3A and 3B are diagrams illustrating a monotonically increasing function used by the statistical language model generation device according to the first embodiment. FIG. 3A illustrates a case where the monotonically increasing function is a power function, and FIG. 3B illustrates a case where the monotonically increasing function is a sigmoid function. , (C) and (d) show the case where the output result of the monotone increasing function has upper and lower limits. In the above description, a monotonically increasing function (power function) having a power parameter of a positive value smaller than 1 as shown in FIG.
The result obtained by substituting the conditional probability of a lower-order word chain calculated at the time of back-off smoothing into a new conditional probability of the word chain (for example, f (P) = a
P ^b , b is a power parameter).

On the other hand, in the apparatus of the present invention, if the monotone increasing function is f, for example, the sigmoid function as shown in FIG. 3B and the upper and lower limits as shown in FIGS. 3C and 3D are limited. You may use the function provided. The sigmoid function (f (P) = 1 / (1+
e− ^{α (P + β)} ), α> 0) is one of the non-linear functions, and the degree of non-linearity can be changed by appropriately changing the value of the parameter α. , The smaller the size, the greater the nonlinearity). Thereby, the range in which the conditional probability can be taken can be set in a wide range by appropriately changing the parameter α, and the processing is flexibly performed so that the dynamic range due to the frequency difference between the target word chains becomes an appropriate value. can do. Further, if the possible values shown in FIGS. 3C and 3D are functions whose upper and lower limits are limited, only words having a high appearance frequency such as case particles have the effect of compressing the probability. In other words, words having a high appearance frequency, such as case particles, also have an increased estimated value of the conditional probability. However, by associating the estimated value with the output value of the function, the upper limit value is forcibly set as the estimated value. Such a function whose upper and lower limits are limited can be realized by providing a parameter for limiting the output value to the sigmoid function shown in FIG. In the example of FIG. 3C, a parameter that limits the output value is provided in the sigmoid function, and the parameter α can be realized by increasing the value of the parameter α.
In the example, the parameter can be realized by providing a parameter for limiting the output value to the sigmoid function and reducing the value of the parameter α.

Here, specific examples of conditional probabilities of word chains when the above monotone increasing function f is used are shown in the following equations (20) and (21).

(Equation 3) However, since f and g are instead of probabilities, the following equation (22),
The normalization condition shown in (23) must be satisfied.

(Equation 4) By using the monotonically increasing functions f and g as described above, the same effect as in the case of substituting the probability into the power function in the above embodiment can be obtained.

In the first embodiment, Witten
-The smoothing by the Bell method has been described as an example, but it goes without saying that the same effect can be obtained by another conventional smoothing method.

As described above, according to the first embodiment, when performing the back-off smoothing to calculate the conditional probability of a word chain having a small frequency of appearance (a value as close to 0 as possible, 0), Since the off-probability is input to a function that monotonically increases, and the result of this function is used as the new conditional probability of word chains, it is possible to prevent word chains that include words with high frequency of occurrence from having an unduly high language probability. Can be.

Further, according to the first embodiment, a power function, a sigmoid function, and a calculation result having upper and lower limits are used for a function that increases monotonically. The processing can be performed so that the dynamic range based on the frequency difference between the word chains to be executed becomes an appropriate value.

Embodiment 2 The second embodiment shows a configuration for determining the power (or monotonically increasing function) parameter in the first embodiment.

FIG. 4 is a diagram showing a configuration of a statistical language model generation device according to the second embodiment of the present invention. In the figure, reference numeral 7 denotes a corpus dividing means for dividing the language corpus into a plurality of pieces. For example, texts in the language corpus stored in the corpus storage means 1 are arranged in time series so as to be divided in correspondence with each series, or the text is divided at random. Select to divide. Numeral 8 denotes a parameter setting means for classifying a plurality of divided language corpuses into a language corpus for frequency counting and a language corpus for evaluating language performance by the word chain frequency counting means 3 and a word connection. For example, a power parameter of the monotone increasing function used by the probability recalculation means 5 is set. Reference numeral 9 denotes a perplexity calculating means (language performance evaluation means) which uses a conditional probability of a word chain calculated in a language corpus for evaluating perplexity (perplexity) as an index for evaluating language performance. calculate. Numeral 10 is an optimal parameter determining means for determining an optimal parameter of a monotonically increasing function that gives the highest perplexity. Each time the parameter setting means 8 changes the language corpus classification and changes the parameters of the monotonically increasing function, parsing is performed. The perplexity value calculated by the plexity calculating means 9 is detected, and the parameter of the monotonically increasing function when the perplexity becomes the minimum value is determined as the optimum parameter. Although the language model generating means 6 is omitted in the illustrated example, it is assumed that the language model generating means 6 is actually provided, and the word connection probability calculating means 4 and the word A statistical language model is generated using the conditional probability of the word chain calculated by the connection probability recalculating means 5. The same components as those in FIG. 1 are denoted by the same reference numerals, and duplicate description will be omitted.

Next, the operation will be described. FIG. 5 is a flowchart showing a statistical language model generation operation by the statistical language model generation device of FIG. 4. An example in which a trigram language model is created along this flowchart will be described. First, as in the first embodiment, the corpus input unit 2 reads the language corpus from the corpus storage unit 1, and the corpus dividing unit 7 holds the corpus available (step ST1a).

Next, the corpus dividing means 7 divides the language corpus read into the corpus input means 2 into two corpora, which are referred to as language corpuses C ₁ and C ₂ (step ST2).
a, parameter setting step). As a method of division,
As described above, there are a method in which texts in a language corpus are arranged in time series and divided according to each series, and a method in which text is randomly selected and divided.

[0065] When the corpus dividing means 7 divides the corpus, one of the language corpus parameter setting unit 8 is divided (here, assumed as C ₁₎ and a frequency counter according to the word chain, other Is classified for language performance evaluation (language corpus C ₂ ). Thereafter, word frequency of triads in the word chain frequency counting unit 3 is in the corpus C ₁ classified as a counting frequency N
(W ₁ , w ₂ , w ₃ ), the appearance frequency N (w
₁ , w ₂ ), word appearance frequency N (w ₁ ), and type frequency R (w ₁ ,
w _2), counts the R _{(w 1)} (step ST3a, word concatenation frequency counting step).

At this time, the parameter setting means 8 sets an appropriate value parameter for the monotone increasing function described in the first embodiment (step ST4a, parameter setting step). This parameter setting is performed by automatically reading and setting a plurality of predetermined parameter values stored in advance in a memory (not shown), or setting a desired value by a user using input means (not shown). There is a method such as doing.

Next, the perplexity calculating means 9 converts the language
Corpus C₂Calculate perplexity PP for
You. Here, the perplexity PP will be described.
Perplexity PP measures the performance of statistical language models
Sentence S = w₁w₂w₃... w_nof
Appearance probability P (w₁w₂w₃...) is the following formula (2)
The relationship is as shown in 4). Specifically, a language that follows
The number of trials and errors required to determine the word
The average number of branches is perplexity PP, and this value is
It is said that the smaller the language model, the better the performance of the language model. PP = P (w₁w₂w₃... w_n)^{− (1 / n)} ... (24) In the case of the trigram model, the following equation (25)
Is approximated by P (w₁w₂w₃... w_n) = P (w₁) ・ P (w₂| W₁) ・ P (w₃| W₁w₂) ・ P (w₄| W₂w ₃ ) (25) This P (w₃| W₁w₂), P (w₄| W₂w₃),
Is calculated by the word connection probability calculation means 4 and the word connection probability.
Equation (16) shown in the first embodiment by the rate recalculating means 5
To (19) (or equations (20) to (23))
(Step ST5a, language performance evaluation step). S
The perplexity PP obtained in Step ST5a is
It is output to the appropriate parameter determining means 10.

Next, the parameter setting means 8 sets the parameter of the monotone increasing function to another value, and the perplexity calculating means 9 calculates the perplexity PP. The operation up to this point is repeated for some parameter values (step ST6a, language performance evaluation step). Thereby, the perplexity P corresponding to each parameter is obtained.
P is stored in the optimum parameter determination means 10.

The optimal parameter determining means 10 calculates the perplexity PP calculated for some parameter values.
Are held, the parameter value that gives the smallest (high performance) perplexity PP is determined as the parameter 1 (step ST7a, optimum parameter determination step).

[0070] Further, by changing the respective classification of corpus classified into a parameter setting means 8 for and the language performance evaluation frequency counting classified in step ST3a, the corpus C ₂ and a frequency counter, corpus the C ₁ and language performance for evaluation. When classification is changed, counts the word concatenation frequency the same process in corpus C _2, to calculate the perplexity PP with corpus C _1, the parameter value which gives the smallest perplexity PP Determined as parameter 2 (step ST8a to step ST12a, parameter setting step, language performance evaluation step, optimal parameter determination step).

When the above operation is completed and the parameters 1 and 2 that give the smallest perplexity PP in each pattern are obtained, the optimum parameter determining means 10 calculates the average value of the parameters 1 and 2 to obtain the final optimum parameter. (ST13a).

As described above, according to the second embodiment, the language corpus is divided into a plurality of parts, and the language corpus is classified into a language corpus for frequency counting and a language corpus for language performance evaluation. The parameter of the increasing function is set, and the conditional probability of the word chain in the language corpus for language performance evaluation is calculated based on the frequency of the word chain counted in the language corpus for frequency counting. The value of the perplexity PP which is an index of the language performance is obtained using the above-mentioned parameters, and these operations are repeated to obtain a parameter that gives the smallest perplexity PP, and the average value of these parameters is determined as an optimal parameter. The linguistic performance used by the word connection probability recalculation means 5 of the first embodiment monotonically increases so as to generate the statistical language model having the best language performance. It is possible to set the number of parameters.

Note that, in addition to those described in the second embodiment, the corpus dividing means 7 divides the language corpus into three or more parts, and includes a language corpus for counting the frequency of word chains and a language performance evaluation part. Is classified into a language corpus for calculating the perplexity PP of each of the combinations, different parameters are set for each combination, and a parameter that gives the highest language performance (smallest) perplexity PP for each combination is detected. The average value of these parameters can be used as the final optimum parameter. Also according to this, the same effect as in the second embodiment can be obtained.

Embodiment 3 The third embodiment is intended to prevent a deterioration in the accuracy of the word chain probability (conditional probability of the word chain) due to the use of the class language model, and to provide a high-accuracy language model.

FIG. 6 is a diagram showing a configuration of a statistical language model generation device according to the third embodiment of the present invention. In the figure, reference numeral 11 denotes class information storage means (word class storage means).
And stores the class information of the words existing in the language corpus. The class information is information that classifies words having similar meanings in the language corpus into classes and associates the classes with the corresponding words. Reference numeral 12 denotes a class information input unit, which reads out class information from the class information storage unit 11 and stores the class information so that the similar word chain adding unit 13 can use it. Reference numeral 13 denotes a similar word chain adding unit that newly adds a word chain similar to a word chain existing in the language corpus to the language corpus. Reference numeral 14 denotes an additional corpus, and a word that is newly similar by the similar word chain adding unit 13. The chain is equivalent to the added language corpus. Also,
Although the word connection probability calculation means 4, the language model generation means 6, or the word connection probability recalculation means 5 is omitted in the illustrated example, the word connection probability calculation means 4, the language model generation means 6, or the word connection probability It is assumed that the language model generating unit 6 includes a connection probability recalculating unit 5, and a language model generating unit 6 generates a statistical language model using the conditional probability of the word chain calculated by the word connecting probability calculating unit 4. FIG.
The same components as those described above are denoted by the same reference numerals, and redundant description will be omitted.

Next, the operation will be described. 7, 8,
9 is a flow chart showing a statistical language model generation operation by the statistical language model generation device of FIG. 6, and a case where a trigram language model is created along this flow chart will be described as an example. First, as in the first embodiment,
The corpus input means 2 reads the language corpus from the corpus storage means 1, and holds the word chain frequency counting means 3 and the similar word chain adding means 13 in a usable state (step ST).
1b).

Next, the word chain frequency counting means 3 counts the frequency related to word chains from the language corpus read into the corpus input means 2 (step ST2b, word chain frequency counting step). Specifically, the word chain frequency counting means 3 calculates the appearance frequency N of the three chains of words in the language corpus.
(W ₁ , w ₂ , w ₃ ), the appearance frequency N (w
₁ , w ₂ ) and word appearance frequency N (w ₁ ) are calculated.

Thereafter, the class information input means 12 reads the class information from the class information storage means 11 and holds the similar word chain adding means 13 usable (step S).
T3b). The classification may be an intuitive manual classification or a mechanical classification based on a certain standard. Also, not all words need to belong to any class. Furthermore, the class to which a certain word belongs is not necessarily one, but may belong to a plurality of classes. FIG. 10 is a diagram showing an example of class information used in the statistical language model generation device according to the third embodiment. In the illustrated example, the medicines are classified according to their types and effects, and words in the language corpus are classified into classes 1 and 2, respectively.
Are managed in association with 2, 3,.... In addition, this class information is an example of intuitive classification by hand in the field related to medical treatment. Specifically, the class information is classified into classes such as topical medicines, oral medicines, and abdominal diseases. Words related to 1, 2, 3,... Are assigned.

Next, the similar word chain adding means 13 outputs
-Any word chain w present in the path ₁-W₂Select one
I do. This two-word chain w₁-W₂The word w following
₃Are usually of multiple types. After this, add similar word chains
Means 13 selects a certain class C and belongs to this class C
The word to be w₁-W₂Find out if there is anything following
(Step ST4b). Someone belonging to this class C
The word x is w₁-W₂(That is, N
(W₁, W₂, X)> 0), other units belonging to class C
If the word y is N (w₁, W₂, Y) = 0
N (w₁, W₂, Y)
Is given a predetermined appearance frequency k (step ST5b, similar
Word chain addition step).

In the above-described operation, the word chain that would appear if the language corpus was made very large (a large amount of text was held) was reduced to an integer frequency 0 because the language corpus was small. This is for preventing a problem caused by the above. Accordingly, 0 <k ≦ 1 is appropriate as the value of the appearance frequency k. For example, class 1
If the word chain of “applied medicine” in the language corpus includes w ₁ = “every day”, w ₂ = “evening”, and x = “powder”, it is in the same class 1 as “powder” shown in FIG. Even if there is no word chain "everyday"-"evening"-"vaseline" by a certain word "vaseline", its appearance frequency is given as k. In this way, the similar word chain adding unit 13 gives a frequency of appearance to the similar word chain and newly adds it to the language corpus, and holds it as the additional corpus 14.

Thereafter, the similar word chain adding means 13 performs the above processing for all classes C and all two-word chains w ₁ -w ₂ (step ST6b, step ST7).
b, similar word chain addition step).

Next, the similar word chain adding means 13 outputs
-A word chain w in the path ₁-*-W₃choose
I do. Here, * indicates any unit existing between three word chains.
Word, usually there are multiple types of words. After this, similar
The word chain adding means 13 selects a class C, and this class
The word belonging to Lass C is w₁-*-W₃In *
Is checked (step ST8b). In this class C
A certain word x belongs to w₂-*-W₃Exists in *
(Ie, N (w₂, X, w₃)> 0), class C
Is another word y belonging to N (w₁, Y, w₃) = 0
Considers that it may be possible to connect to it,
(W₁, Y, w₃) Is given a predetermined appearance frequency k (step
Step ST9b, similar word chain addition step). like this
The similar word chain adding means 13 sets the similar word chain
To the language corpus with the frequency of appearance
Stored as corpus 14.

Thereafter, the similar word chain adding means 13 performs the above-described processing for all classes C and all combinations of w ₁ and w ₃ (step ST10b, step ST10).
11b, similar word chain adding step).

[0084] Next, similar word concatenation additional means 13 selects a certain word concatenation * _-w 2 -w ₃ present in the corpus. Here, * is an arbitrary word having a three-word chain, and usually includes a plurality of types of words. Thereafter, select the class C which have similar word concatenation additional means 13, investigate whether words belonging to this class C is present in the * _-w 2 _{-w 3} * (step ST12b). If there is word x that belong to the class C was present in the * _-w 2 _{-w 3} * _{(ie, N (x, w 2,} w 3)> 0), other word of belonging to the class C Even if y is N (y, w ₂ , w ₃ ) = 0, it is considered that there is a possibility that it can be connected to N (y, w ₂ , w ₃ ).
w ₂ , w ₃ ) is given a predetermined appearance frequency k (step S
T13b, similar word chain addition step). In this way, the similar word chain adding unit 13 gives a frequency of appearance to the similar word chain and newly adds it to the language corpus, and holds it as the additional corpus 14.

Thereafter, the similar word chain adding means 13 performs the above-described processing for all combinations of classes C and all combinations of w ₂ -w ₃ (step ST14b, step ST14).
15b, similar word chain addition step).

As described above, according to the third embodiment, a word chain similar to the word chain existing in the language corpus is newly added to the language corpus. Specifically, the words existing in the language corpus are added. Based on the classified class information, it is determined whether a word belonging to a predetermined word class is included in a word chain in a language corpus, and a word belonging to the predetermined word class is replaced with another word belonging to this word class. Is added to the language corpus as a similar word chain to the language corpus. Even if the frequency of occurrence is 0, a given frequency of appearance is given to reduce the sparseness. Because it is possible, the case of backing off when calculating the language probability is reduced, and the performance of the statistical language model can be improved. In the third embodiment, the class information is used only when a similar word chain is created, and is not used when calculating a word chain probability (conditional probability of a word chain). Deterioration can be prevented.

A similar effect is obtained even if, for example, steps ST8b to ST15b, which are partial processes, are omitted from the processes described in the third embodiment.

Embodiment 4 The fourth embodiment is different from the third embodiment in that the use of a class language model prevents the word chain probability (conditional probability of word chain) from deteriorating in accuracy and provides a language model with high accuracy. It is.

FIG. 11 is a diagram showing a configuration of a statistical language model generation device according to the fourth embodiment of the present invention. In the figure, reference numeral 15 denotes word-class chain counting means (inter-class frequency counting means), which indicates the frequency of occurrence of word chains present in the language corpus read from the corpus storage means 1 by the corpus input means 2. Counting is performed for each word class to which the word located at the end of the chain belongs, and the average value of the appearance frequency based on the number of word types in this word class is calculated.
Reference numeral 16 denotes a word connection probability calculating unit that calculates the conditional probability of a word chain based on the frequency of the word chain counted by the word chain frequency counting unit 3 and the word counted by the word-class chain counting unit 15. Calculate the conditional probabilities of the word class to which the word located at the end of the word chain belongs based on the average value of the frequency of occurrence of the chain, and calculate the probability obtained by linearly combining these with the new conditional probability of the word chain. I do. Reference numeral 17 denotes coefficient storage means for storing a weight coefficient added to the conditional probability that the word connection probability calculation means 16 is linearly combined. The same components as those in FIGS. 1 and 6 are denoted by the same reference numerals, and redundant description will be omitted.

Next, the operation will be described. FIG. 12 shows FIG.
FIG. 4 is a flowchart showing a statistical language model generation operation by one statistical language model generation device, and a case where a trigram language model is created along this flowchart will be described as an example. First, as in the first embodiment, the corpus input means 2 reads the language corpus from the corpus storage means 1, and holds the word chain frequency counting means 3 and the word-class chain counting means 15 in a usable state (step ST1c). .

Next, the word chain frequency counting means 3 counts the word chain frequency from the language corpus read by the corpus input means 2 (step ST2c, word chain frequency counting step). Specifically, the word chain frequency counting means 3 calculates the appearance frequency N of the three chains of words in the language corpus.
(W ₁ , w ₂ , w ₃ ), the appearance frequency N (w
₁ , w ₂ ) and word appearance frequency N (w ₁ ) are calculated.

Thereafter, the class information input means 12 reads the class information from the class information storage means 11 and reads the class information.
Class chain counting means 15 and word connection probability calculating means 16
Are kept available (step ST3c). The classification may be an intuitive manual classification or a mechanical classification based on a certain standard. Also, not all words need to belong to any class. Furthermore, the class to which a certain word belongs is not necessarily one, but may belong to a plurality of classes.

Next, the word-class chain counting means 15 counts the number of chains of the word and the succeeding word for each class C, and obtains N (w ₁ , w ₂ ,
C), N (w ₂ , C) and N (C) are obtained (step S)
T4c).

(Equation 5) For example, w ₁ = “today”, w ₂ = “is”, class C ₁ is class 1 (applied medicine) in FIG. 10, and “... today is a powder ...” twice in the language corpus. , "...
・ Vaseline ... 4 times today, “… Linderon ... 4 times today,” ... Vantelin at night ...
・ If there is a word chain of “3 times” and “... Vanterin ...” once, N (“Today”, “ha”, “
C ₁ ) = 2 + 4 + 4 = 10, N (“is”, C ₁ ) = 2 +
4 + 4 + 3 = 13, N (C ₁ ) = 2 + 4 + 4 + 3 + 1 =
It becomes 14.

Further, the word-class chain counting means 15
Is a simple expression in the class C of the appearance frequency obtained as described above.
Average value N based on number of word_AV(W₁, W₂, C), N
_AV(W ₂, C), N_AV(C) according to the following equation (27)
(Step ST5c).

(Equation 6) In the above example, word 1 belonging to the class C ₁
0, and there are five types, N _AV (“today”, “ha”,
C ₁ ) = 10/5 = 2, N _AV (“ _ha ”, C ₁ ) = 13
/5=2.6 and N _AV (C ₁ ) = 14/5 = 2.8. The above steps ST4c and ST5c correspond to a word class frequency counting step.

Next, the word connection probability calculation means 16 reads the coefficients λ ₁ to λ ₆ from the coefficient storage means 17 (step S).
T6c). The coefficients λ _{1 to} λ ₆ are λ ₁ + λ ₂ +... + Λ ₆ = 1 due to the restriction that the sum of probabilities = 1. The coefficients λ _{1 to} λ ₆ can be determined by a well-known EM algorithm. The EM algorithm is a method of obtaining an appropriate convergence value of a parameter by a recurrence formula starting from an initial value. Here, assuming that the k-th coefficient λ _j in the recurrence equation is λ ^k _j , the (k + 1) _-th coefficient λ ^{k + 1} _j is given by the following equation (28).

(Equation 7) Starting from λ ¹¹ ₁ as the initial value of the coefficient λ, the above equation (2)
The calculation according to 8) is continued until the coefficient λ converges. In this equation (28), the first Σ is taken for all word chains appearing in the language corpus. N is the total frequency of all word chains appearing in the language corpus.

Next, the word connection probability calculation means 16 extracts a word chain w ₁ w ₂ w ₃ composed of any three words,
The word chain probability P (w ₃ | w ₁ , w ₂ ) is calculated as follows. First, based on the class information read by the class information input unit 12 from the class information storage unit 11,
Word connection probability calculation unit 16 obtains the class to which the word _{w 3} located at the end of the word concatenation _w 1 _w 2 _{w 3} belongs, which is referred to as Class _{C 3} (step ST7c).

Thereafter, the word connection probability calculating means 16 calculates
The word chain probability P (w ₃ | w ₁ ,
w ₂ ) (step ST8c, word connection probability calculation step).

(Equation 8) Here, * means all words. The above equation (2
The coefficients λ _{1 to} λ ₆ of 9) are, as described above, the sum of the probabilities =
Due to the restriction of 1, λ ₁ + λ ₂ +... + Λ ₆ = 1.

The word-class chain counting means 15 and the word connection probability calculating means 16 controlled by the language model generating means 6 perform the above processing on all possible word chains (w
₁ , w ₂ , w ₃ ). As a result, the trigram language model is created while the conditional probability of each word chain is held in the language model creating means 6 (step ST9c, statistical language model creating step).

Here, the above equation (29) will be described. The part of the first to third terms on the right side of the above equation (29) is
Although this corresponds to a conventionally well-known method called linear interpolation, the fourth embodiment on the right side
By adding the terms 6 to 6, the accuracy of the conventional linear interpolation method is improved. For example, when the amount of the corpus stored in the language corpus is small, even if the numerator of the first term on the right side is 0, the class C to which the same word group belongs
_In many cases, the numerator of the fourth term on the right side taking into account ₃ does not become 0. At this time, the word chain probability increases by the amount of the fourth term on the right side. As described above, even when the frequency of appearance is 0 in a language corpus with a small corpus amount, or there is a word chain having a value as close as possible to 0, the fourth to sixth terms on the right side corresponding to the linear interpolation method Can perform the interpolation to estimate an appropriate word chain probability, and as a result, the accuracy of the speech recognition apparatus is improved. Further, in the fourth embodiment, the class information is used only when a similar word chain is created, and is not used when calculating a word chain probability, so that there is no performance degradation associated with the class language model.

As described above, according to the fourth embodiment, the appearance frequency of the word chain existing in the language corpus and the frequency of the word chain including the number of word types are counted, and the number of words existing in the language corpus is counted. Based on the class information obtained by classifying words to be classified into word classes having similar meanings and the frequency of word chains, the frequency of occurrence of word chains existing in the language corpus is changed to the word to which the last word of this word chain belongs. Counting by word class, calculating the average value of the appearance frequency according to the number of word types in this word class, based on the conditional probability of the word chain calculated based on the frequency related to the word chain, and the average value of the occurrence frequency The probability that is obtained by linearly combining the calculated probability with respect to the word class to which the word located at the end of the word chain belongs belongs to the new conditional probability of the word chain. Since it is possible to precisely estimate the word chain probabilities even corpora amount is small because it calculates a word chain probabilities using information classified words into classes.

In the fourth embodiment, an example is shown in which only one type of class information is used, but a plurality of class information may be used. In that case, the word w ₃ belongs to C _a in the first class information, belong to the D _b in the second class information, -
..., the word chain probability P (w ₃ | w ₁ , w
₂ ) is given by the following equation (30).

(Equation 9) Here, λ ₁ + λ ₂ + λ ₃ +... = 1. By using the class information of a plurality of references in this way, there is an advantage that the robustness against the problem that the appearance frequency becomes 0 and the problem of sparsity can be improved.

Further, according to the statistical language model generating apparatus shown in the first to fourth embodiments, a statistical language model having high linguistic performance can be generated. Thus, the accuracy of voice recognition can be improved by configuring a voice recognition device that recognizes input voice.

Further, each means described in the above-described first to fourth embodiments is a hardware /
Can be configured with any software. In the case of using software, a computer-readable recording medium that records a software program that causes a computer to execute the statistical language model generation method of the present invention enables word chains including words with a high appearance frequency to be formed. It is possible to easily realize a computer system functioning as a statistical language model generation device that can prevent having an unduly high language probability or prevent performance degradation accompanying a class language model.

[0104]

As described above, according to the present invention, the frequency of occurrence of word chains existing in the language corpus and the frequency of word chains including the number of word types are counted, and the frequency of this word chain is counted. Calculates the conditional probability of a word chain based on, and calculates the conditional probability of a word chain with a low frequency of occurrence by back-off smoothing. Since the result is a new conditional probability of the word chain, the dynamic range due to the frequency difference between the word chain including the word having a high appearance frequency can be relaxed. There is an effect that it is possible to prevent having an unduly high language probability.

According to the present invention, since the monotonically increasing function is a power function, there is an effect that the dynamic range due to a frequency difference between a word chain including a word having a high appearance frequency can be reduced.

According to the present invention, since the monotonically increasing function is a sigmoid function, the range in which the conditional probability can be taken can be set in a wide range by appropriately changing the parameters. There is an effect that the processing can be flexibly performed so that the dynamic range based on the frequency difference becomes an appropriate value.

According to the present invention, since the monotonically increasing function has upper and lower limits in the operation result, it is possible to reduce the dynamic range due to the frequency difference between a word chain including a word having a high appearance frequency, and There is an effect that the probability can be reduced only for words having a high appearance frequency.

According to the present invention, the language corpus is divided into a plurality of languages, and the plurality of divided language corpuses are classified into a language corpus for counting frequencies and a language corpus for evaluating language performance. Set the parameter of the increasing function to a desired value, calculate the conditional probability of the word chain in the language corpus for language performance evaluation based on the frequency related to the word chain counted in the language corpus for frequency counting, The language performance is evaluated using this conditional probability, and the parameter that gives the best language performance is detected each time the classification of the divided language corpus is replaced and the parameter of the function that monotonically increases is detected. Since the average value is determined as the optimal parameter, the monotonically increasing function is used to generate the statistical language model with the best language performance. There is an effect that it is possible to set the parameters.

According to the present invention, a plurality of divided language corpora are classified into a frequency corpus language corpus and a language performance evaluation language corpus relating to word concatenation. Since different parameters are set for each combination, the parameter that gives the best linguistic performance for each combination is detected, and the average value of these parameters is determined as the final optimal parameter. Can play.

According to the present invention, a word chain similar to the word chain existing in the language corpus is newly added to the language corpus, and the frequency of the word chain existing in the language corpus is counted. Since the conditional probability that words are chained is calculated based on the frequency according to the above, a word having a possibility of chaining is given a predetermined frequency even if the frequency of occurrence is 0, so that sparsity can be reduced. This has the effect of reducing the possibility of back-off during the calculation of the language probability and improving the performance of the statistical language model.

According to the present invention, it is determined whether or not a word belonging to a predetermined word class is included in a word chain in the language corpus based on class information obtained by classifying words existing in the language corpus. Since a word chain obtained by replacing a word belonging to the word class with another word belonging to this word class is newly added to the language corpus as a similar word chain, the same effect as in paragraph 0110 can be obtained. In addition, since class information is used only when creating similar word chains and is not used when calculating conditional probability of word chains, there is an effect that performance degradation associated with a class language model can be prevented. .

According to the present invention, the frequency of occurrence of word chains existing in the language corpus and the frequency of word chains including the number of word types are counted, and words existing in the language corpus are converted into words having similar meanings. Based on the class information classified by class and the frequency related to the word chain, the appearance frequency of the word chain existing in the language corpus is
The conditional probability of the word chain calculated based on the frequency of the word chain, counted by the word class to which the words included in the word chain belong, and the conditional probability of the word class to which the word included in the word chain belongs Is used as the new conditional probability of the word chain.The word chain probability is calculated using the information obtained by classifying the words into classes, so that even when the corpus amount is small, the word chain probability can be accurately calculated. The effect is that it can be estimated.

According to the present invention, a speech recognition apparatus for recognizing an input speech using a statistical language model obtained by the above-described statistical language model generation apparatus is provided, so that speech recognition with improved speech recognition accuracy is provided. There is an effect that the device can be provided.

According to the present invention, a recording medium for recording a program for causing a computer to execute the above-described statistical language model generating method is provided. Therefore, a computer system functioning as a statistical language model generating apparatus having the above-described effects can be easily realized. There is an effect that can be realized.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a statistical language model generation device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a statistical language model generation operation by the statistical language model generation device of FIG. 1;

FIG. 3 is a diagram showing a monotonically increasing function used by the statistical language model generation device according to the first embodiment.

FIG. 4 is a diagram showing a configuration of a statistical language model generation device according to a second embodiment of the present invention.

FIG. 5 is a flowchart showing a statistical language model generation operation by the statistical language model generation device of FIG. 4;

FIG. 6 is a diagram showing a configuration of a statistical language model generation device according to a third embodiment of the present invention.

FIG. 7 is a flowchart showing a statistical language model generation operation by the statistical language model generation device of FIG. 6;

FIG. 8 is a flowchart showing a continuation of the statistical language model generation operation of FIG. 7;

FIG. 9 is a flowchart showing a continuation of the statistical language model generation operation of FIG. 8;

FIG. 10 is a diagram showing an example of class information used in the statistical language model generation device according to the third embodiment.

FIG. 11 is a diagram showing a configuration of a statistical language model generation device according to a fourth embodiment of the present invention.

FIG. 12 is a flowchart showing a statistical language model generation operation by the statistical language model generation device of FIG. 11;

FIG. 13 is a block diagram illustrating a configuration of a conventional speech recognition device that recognizes continuous speech.

14 is an explanatory diagram illustrating generation of a statistical language model used in the speech recognition device in FIG.

[Explanation of symbols]

1 corpus storage means, 2 corpus input means, 3 word chain frequency counting means, 4, 16 word connection probability calculation means, 5 word connection probability recalculation means, 6 language model generation means, 7 corpus division means, 8 parameter setting means, 9 perplexity calculation means (language performance evaluation means), 10 optimum parameter determination means, 11 class information storage means (word class storage means), 12 class information input means, 13 similar word chain addition means, 14 additional corpus, 15 words Class linkage counting means (word class frequency counting means), 17 coefficient storage means.

 ──────────────────────────────────────────────────続 き Continued from the front page (72) Inventor Hiroyasu Goi 2-3-2 Marunouchi, Chiyoda-ku, Tokyo F-term (reference) 5D015 HH23

Claims (20)

[Claims] 1. A corpus storing means for storing a language corpus; a word chain frequency counting means for counting a frequency of a word chain including the frequency of appearance of word chains and the number of word types existing in the language corpus; A word connection probability calculating means for calculating a conditional probability of a word chain based on the frequency of the word chain counted by the word chain frequency counting means; When performing back-off smoothing and calculating, input the back-off probability to a function that monotonically increases,
A statistical language model generating apparatus comprising: a word connection probability recalculating means for setting a calculation result by this function as a new conditional probability of the word chain. 2. The statistical language model generating apparatus according to claim 1, wherein the monotonically increasing function is a power function. 3. The statistical language model generator according to claim 1, wherein the monotonically increasing function is a sigmoid function. 4. The statistical language model generation apparatus according to claim 1, wherein the monotonically increasing function has upper and lower limits in the operation result. 5. A corpus dividing means for dividing a language corpus into a plurality of languages, a language corpus for counting frequencies related to word chains by a word chain frequency counting means, and a language corpus for evaluating language performance. Parameter setting means for setting parameters of a monotonically increasing function used by the word connection probability recalculating means; and the language performance based on the frequency of the word chain counted in the language corpus for frequency counting. A language performance evaluation unit that calculates a conditional probability of a word chain in the language corpus for evaluation, and evaluates the language performance using the conditional probability; Each time the parameter of the monotonically increasing function is changed, the parameter that gives the best language performance 2. The statistical language model generating apparatus according to claim 1, further comprising: an optimum parameter determining means for detecting and determining an average value of these parameters as an optimum parameter. 6. The parameter setting means classifies the plurality of divided language corpus into a language corpus for frequency counting and a language corpus for language performance evaluation, and combines these into a plurality of types of combinations. 6. The method according to claim 5, wherein different parameters are set for the combination of the parameters, and the optimal parameter determining means detects a parameter that gives the best linguistic performance in each combination and determines an average value of these parameters as the optimal parameter. Statistical language model generator. 7. A corpus storage means for storing a language corpus; a word chain frequency counting means for counting a frequency of a word chain including an appearance frequency of a word chain existing in the language corpus and a number of word types; A word connection probability calculating means for calculating a conditional probability of a word chain based on the frequency of the word chain counted by the word chain frequency counting means; and a word chain similar to the word chain existing in the language corpus. A statistical language model generating apparatus comprising: a similar word chain adding means for adding to the language corpus. 8. A word class storage means for storing class information obtained by classifying words existing in a language corpus into word classes having similar meanings, wherein the similar word chain adding means includes a predetermined word based on the class information. It is determined whether the word belonging to the word class is included in the word chain in the language corpus, and the word chain obtained by replacing the word belonging to the predetermined word class with another word belonging to the word class is similar. 8. The statistical language model generation apparatus according to claim 7, wherein a new word chain is newly added to the language corpus. 9. A corpus storage means for storing a language corpus; a word chain frequency counting means for counting a frequency of a word chain including the number of appearances of word chains and the number of word types existing in the language corpus; Word class storage means for storing class information obtained by classifying words existing in the language corpus into word classes having similar meanings; and, based on the class information, an appearance frequency of a word chain existing in the language corpus. The inter-word class frequency counting means for counting for each word class to which the words included in the word chain belong, and the conditional probability of the word chain based on the frequency relating to the word chain, and the inter-word class frequency counting means Based on the calculated appearance frequency of the word chain, the conditional probability of the word class to which the word included in the word chain belongs is calculated. To, statistical language model generator with a word connection probability calculation means for the probability formed by these linear combination with the new conditional probability of the word chain. 10. A speech recognition device for recognizing an input speech using a statistical language model obtained by the statistical language model generation device according to any one of claims 1 to 9. 11. A word chain frequency counting step for counting the frequency of a word chain including the number of appearances of word chains and the number of word types existing in a language corpus; and a word chain frequency counting step. A word connection probability calculation step of calculating a conditional probability of a word chain based on a frequency related to the word chain; and a conditional probability of a word chain having a low frequency of occurrence is subjected to back-off smoothing in the word connection probability calculation step. In the calculation, the back-off probability is input to a function that monotonically increases, and a calculation result by this function is used as a new conditional probability of the word chain. A statistical language model that generates a statistical language model using the conditional probabilities of word chains found in the connection probability recalculation step A statistical language model generation method comprising a generation step. 12. The statistical language model generation method according to claim 11, wherein the monotonically increasing function is a power function. 13. The statistical language model generation method according to claim 11, wherein the monotonically increasing function is a sigmoid function. 14. The statistical language model generation method according to claim 11, wherein the monotonically increasing function has upper and lower limits in the operation result. 15. A language corpus is divided into a plurality of words, and classified into a language corpus for counting frequencies related to a word chain and a language corpus for evaluating language performance, and monotonically increases in a word connection probability recalculation step. A parameter setting step of setting a parameter of the function; and calculating a conditional probability of a word chain in the language corpus for language performance evaluation based on the frequency related to the word chain counted in the language corpus for counting the frequency of appearance. A language performance evaluation step of evaluating language performance using the conditional probability; and obtaining the parameter that gives the highest language performance each time the parameter setting step and the language performance evaluation step are repeated, and averaging these parameters. An optimal parameter determining step of determining a value as an optimal parameter. Statistical language model generating method according to claim 11. 16. In the parameter setting step, the plurality of divided language corpuses are classified into a language corpus for counting frequencies and a language corpus for evaluating language performance related to a word chain, and these are classified into a plurality of types of combinations. Along with
Different parameters are set for each combination, and in the optimal parameter determination step, the parameters that give the best linguistic performance for each combination are detected, and the average value of these parameters is determined as the final optimal parameter. The method for generating a statistical language model according to claim 15, wherein 17. A similar word chain adding step of newly adding a word chain similar to a word chain existing in the language corpus to the language corpus, an appearance frequency and a number of word types of the word chain existing in the language corpus. A word chain frequency counting step of counting the frequency of the word chain including: a word connection probability of calculating a conditional probability of the word chain based on the frequency of the word chain counted in the word chain frequency counting step. A statistical language model generation method, comprising: a calculation step; and a statistical language model generation step of generating a statistical language model using the conditional probability of word chains obtained in the word connection probability calculation step. 18. A word belonging to a predetermined word class is added to the language corpus based on class information obtained by classifying words existing in the language corpus into word classes having similar meanings in the similar word chain adding step. Judge whether it is included in the word chain, and add a word chain obtained by replacing a word belonging to the predetermined word class with another word belonging to this word class as a similar word chain to the language corpus. 18. The statistical language model generation method according to claim 17, wherein: 19. A word chain frequency counting step for counting the frequency of a word chain including the number of occurrences of word chains and the number of types of words existing in the language corpus, and a similar meaning for words existing in the language corpus. A word class frequency counting step of counting the appearance frequency of word chains present in the language corpus for each word class to which the words included in the word chain belong, based on the class information classified for each word class. The conditional probability of the word chain is calculated based on the frequency related to the word chain, and the word to which the word included in the word chain belongs based on the frequency of appearance of the word chain counted in the inter-word class frequency counting step. Calculate the conditional probabilities related to the classes, and use the probability obtained by linearly combining them as the new conditional probability of the word chain. A statistical language model generation step comprising: a word connection probability calculation step; and a statistical language model generation step of generating a statistical language model using the conditional probability of the word chain obtained in the word connection probability calculation step Method. 20. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the statistical language model generation method according to any one of claims 11 to 19.