JPH11110385A - Device and method for processing language - Google Patents

Abstract

PROBLEM TO BE SOLVED: To deepen the degree of relation for a classified word as well and to improve the accuracy of chain probability by semantically classifying the word according to a thesaurus. SOLUTION: A processing block 3 divides the pair of input sentences into respective three parts and by exchanging these parts, a non-semantic sentence is generated. A morpheme analytic block 2 divides the pair of generated non- semantic sentences for the unit of a word. A storage block 4 stores the thesaurus. A word classifying block 6 converts a word to a class code based on the thesaurus. A storage block 8 stores the table of trigram probability between classes. A processing block 10 calculates Perplexity based on the class trigram concerning the set of all the generated non-semantic sentences and selects the set of non-semantic sentences having the highest Perplexity. Thus, the classification of the thesaurus is stored and according to this classification of the thesaurus, the word is classified.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a language processing apparatus and method for selecting / generating an optimum word string in natural language processing, and more particularly to using statistical information between word classes set based on a thesaurus. The present invention relates to a language processing device and method.

[0002]

2. Description of the Related Art Conventionally, a word n-gram model using statistical chain information of n words has been used in a language processing apparatus. In addition, a class n-gram model that creates a word class by classifying words for each part of speech and uses statistical chain information between these word classes (between parts of speech) has also been used. See "Spoken Language Processing" Morikita Publishing. In these systems, for example,
In the case of the word n-gram model, the number of words to be handled increases, and n
Becomes large, a problem called the sparse problem arises, in which the chain probability between words cannot be sufficiently estimated. On the other hand, the class n-gram model between parts of speech has a drawback that only rough language processing can be performed.

[0003]

In an auditory psychological experiment or the like, an experiment is performed to determine whether or not the subject can hear the voice properly. There is a request to check only. As a method for satisfying such a purpose, there is a method of creating a grammatically correct but meaningless sentence (for example, Kazuo Watanabe: "Effects of speech speed delay due to speech speed conversion on elderly hearing-impaired", Japanese Otona The Journal of the Society of Throat Science, Vo
l. 99 pp. 445-453 (1996)). However, no good way to create meaningless test sentences has not been found, and it was virtually impossible to remove the author's arbitrary elements from the prepared test sentences.

On the other hand, in speech recognition, in order to improve the recognition performance, a linguistically likely word string is selected from the recognition results at the acoustic level, and the subsequent recognition is performed using the recognition results obtained so far. In order to reduce the number of possible candidates as a result, the word n-gram is used. In particular, in large vocabulary continuous speech recognition, use of the word n-gram is indispensable. In natural language processing other than speech recognition, the word n-gram is used to select a likely word string from a plurality of word strings. However, in the word n-gram model, for example, n = 3 (chain model of three words: trigram model)
As an example, if the number of words is 20,000 words, it is necessary to consider combinations of 20,000 to the third power (8,000,000,000,000,000,000). In addition to the need for memory, there is a problem that the statistics cannot be estimated from available language databases (this problem is called the sparseness problem). This problem becomes more pronounced when n is greater than 3. Such a problem has a great effect on the processing result in speech recognition and the like.

Accordingly, an object of the present invention is to improve the accuracy of the chain probability between words, rather than classifying words for each part of speech,
It is an object of the present invention to provide a language processing device and a method capable of relaxing the restriction on the memory of the language processing device.

[0006]

In order to achieve the above object, the invention of claim 1 classifies a plurality of words into a plurality of classes in which one class is composed of a plurality of word sets. A language processing device that acquires a linkage probability indicating the degree of linkage between the classified classes and selects a class and / or a word string based on the acquired linkage probability, and a storage unit that stores the thesaurus classification. Classification means for classifying the plurality of words according to the classification of the thesaurus.

According to a second aspect of the present invention, in the language processing apparatus according to the first aspect, the classification means converts a word into a classification code when classifying the plurality of words, and converts the word into a classification code when calculating the chain probability. It is characterized in that a word string is calculated in the form of a classification code string.

According to a third aspect of the present invention, in the language processing apparatus according to the second aspect, the classifying means uses, as a classification code, the attached word itself for an attached word attached to the word.

According to a fourth aspect of the present invention, in the language processing apparatus according to the second aspect, when the word is an independent word that is not described in the thesaurus,
It is characterized by classifying each part of speech.

According to a fifth aspect of the present invention, there is provided the language processing apparatus according to the first aspect, further comprising a selection unit for selecting the word sequence having the smallest chain probability as a meaningless sentence used in a hearing experiment. Features.

According to a sixth aspect of the present invention, there is provided the language processing apparatus according to the first aspect, further comprising a selecting means for selecting a word string having the largest chain probability as an optimal candidate among speech recognition candidates. Features.

According to a seventh aspect of the present invention, in the language processing apparatus according to the first aspect, an occurrence probability of a word in a class and a chain probability between classes are obtained in relation to calculation of a chain probability between words. If the chain probability between words cannot be directly obtained, the chain probability between the words is estimated based on the occurrence probability of the word and the chain probability between the classes.

According to an eighth aspect of the present invention, a plurality of words are classified into a plurality of classes in which one class is composed of a plurality of words, and a linkage probability indicating a degree of linkage between the classified classes. In a language processing method for selecting a class and / or a word string based on the acquired chain probability, storing classification of a thesaurus, and classifying the plurality of words according to the classification of the thesaurus It is characterized by.

According to a ninth aspect of the present invention, in the language processing method according to the eighth aspect, when the plurality of words are classified, the words are converted to a classification code, and when the linkage probability is calculated, the word string is converted to a classification code. The calculation is performed in the form of a column.

According to a tenth aspect of the present invention, in the linguistic processing method according to the ninth aspect, for the attached word attached to the word, the attached word itself is used as a classification code.

According to an eleventh aspect of the present invention, in the language processing method according to the ninth aspect, if the word is an independent word that is not described in the thesaurus classification, classification is performed for each part of speech. I do.

According to a twelfth aspect of the present invention, in the linguistic processing method according to the eighth aspect, the word sequence having the smallest chain probability is selected as a meaningless sentence used in a hearing experiment.

According to a thirteenth aspect of the present invention, in the linguistic processing method according to the eighth aspect, the word sequence having the maximum chain probability is selected as an optimal candidate among the speech recognition candidates.

According to a fourteenth aspect of the present invention, in the linguistic processing method according to the eighth aspect, an occurrence probability of a word in a class and a chain probability between the classes are obtained in relation to calculation of a chain probability between words. If the chain probability between words cannot be directly obtained, the chain probability between the words is estimated based on the occurrence probability of the word and the chain probability between the classes.

[0020]

Embodiments of the present invention will be described below in detail with reference to the drawings.

First, a language processing method to which the present invention is applied will be described. In the present embodiment, a class n-gram model based on a thesaurus is used. The word n-gram model is a model that approximates the occurrence of a word string by an n-1 double Markov process. Therefore, given a word string w ₁ w ₂ ... W _m−1 , the probability of occurrence of word w _m is given by

[0022]

(Equation 1)

(Where m ≧ n). Now, it is assumed that each word belongs to only one class.
A class that word w _i belongs represented by c _i. At this time, in the class n-gram model, the probability of occurrence of the word w _m is approximated by the following equation under the condition that the word string w ₁ w ₂ ... W _m-1 is given:

[0024]

(Equation 2)

In generating a test sentence for the psychoacoustic experiment, an equation obtained by simplifying the equation 2 is:

[0026]

[Number 3] _{P (c m | c m c} m-1) is also available.

The class n-gram model classifies words into classes and uses the probability (chain probability) indicating the degree of linkage between classes and the appearance probability of words in the class. It is possible to greatly reduce the number of parameters as compared with. For example, if the vocabulary size (the number of words) to be handled is V and the number of classes is C, the word n
The number of independent parameters in -gram is V ⁿ -1
It is. On the other hand, the number of independent parameters of the class n-gram model is C ⁿ -1 + VC. V = 2,00
Table 1 shows the number of independent parameters when 0 and C = 1,000.

[0028]

[Table 1]

As a class n-gram model, a method of setting a word class based on a part of speech has been conventionally proposed, but there is a problem that only rough word classification can be performed. Further, a method of classifying words using mutual information between words has also been proposed, but there is a known problem that there is no algorithm for finding an optimum classification. On the other hand, a thesaurus is known as a method for performing semantic classification of words. Therefore, in the embodiment, a problem in the word n-gram model, such as the sparseness problem, is achieved by using the class n-gram model based on the thesaurus.

For this reason, an example in which a meaningless sentence for a listening test for an auditory psychological experiment is generated (first embodiment) An example of handling a sparseness problem in a word n-gram model for speech recognition ( (2nd Embodiment) The following two cases are exemplified. In each case, n-
The case where n of the gram is set to 3 (trigram (linkage probability of three sets)) is shown.

In the first and second embodiments, examples are shown in which a classification vocabulary table of the National Institute for Japanese Language is used as a thesaurus. In the classification vocabulary table, five-digit numbers separated by periods, such as 1.3062, are given as code numbers of the respective words. If this code number is directly used as a number representing one class, the number of classes becomes too large. Therefore, in the present embodiment, after truncating the last digit, a set of words having the same code number is replaced with one class. Treat as

Since only the independent words are basically registered in the classification vocabulary table, handling of attached words becomes a problem when using the class model of the formula (2). Another problem is how to handle words that are not registered in the classification vocabulary table.
Although it is possible to compose a class by grammatically classifying the attached words, in the first and second embodiments, a method of constituting one class with one word is adopted.
For words that are not registered in the classification vocabulary table,
In class model learning, a method of automatically registering based on the morphological analysis results was adopted.

In the learning of the word n-gram, usually, a sentence in a language database is divided into words by morphological analysis, and then the probability value of Equation 1 is estimated from the frequency of word chains. Class n-gram and learning are performed in the same procedure.

First, the language database is divided into words, and the words are converted into class codes by collating with the classification vocabulary table. At this time, the attached word is not converted into a code, and the word itself is used as a class code. For independent words not described in the classification vocabulary table, the following processing is performed. First, the inflected forms such as verbs are collated with the stem, and then the classification code numbers corresponding to the inflected forms are added to a separately prepared word-class correspondence table. After forcibly assigning the code of a numeral to the numeral, the numeral is also added to the word-class correspondence table. For nouns, according to the results of morphological analysis, "common nouns", "sa-variable nouns", "proper nouns"
The words are classified into “formal noun”, “adverbial noun”, and “temporal noun”, and are added to a word-class correspondence table (table indicating the correspondence between classes and words included in the class). Other words are classified into unknown classes.

For learning the word trigram and the class trigram, a news manuscript database of a broadcasting station was used. This was from 1992 to May 1996,
It is a database of news manuscripts created by reporters of broadcasting stations. In class trigram learning, the vocabulary size was 20,000 words, and the other words were assigned to a class dedicated to unknown words. This class for unknown words is different from the class for unknown words (the former is a class of words that are not included in the vocabulary, and the latter is a word that is included in the vocabulary range. Is a set). The number of obtained classes was 936, of which 518 classes were created from the classification vocabulary table. The average number of words assigned to each class for independent words was 41. On the other hand, a word trigram is created under the same conditions.

(First Embodiment) Hereinafter, an example will be described in detail. When conducting a listening test of a voice, in order to check whether or not the voice was heard, it is conceivable to create an incomprehensible sentence consisting of plain words and perform an experiment using the voice read out of this sentence. Can be Hereinafter, generation of such a meaningless sentence will be exemplified.

A meaningless sentence is created by dividing a short meaningful sentence prepared in advance into three parts and exchanging these parts. An example of a meaningful sentence is shown below.

There is no smoking on motorcycles. My teeth hurt this morning. The drawer is locked.
A pigeon stopped on his head. There are many tall buildings.

However, for the sake of simplicity, each sentence is converted into a class code string using a classification vocabulary table and a word-class correspondence table, and each sentence is converted using only the trigram probability between the classes in Equation (3). Is calculated, and a sentence set with the lowest probability is selected to create a meaningless sentence. Further, in order to show the effect of the present invention by comparison, similar processing was performed using a word trigram. In the following, for convenience, the result is displayed using Perplexity instead of the probability value. Perplexity in this case indicates the occurrence probability of each sentence (represented by a word string w ₁ ... W _n ) as P (w ₁ .
_n ), its entropy

[0040]

(Equation 4)

Then,

[0042]

## EQU5 ## PP = 2 An amount represented by ^{H (L)} . Therefore, the smaller the occurrence probability of a sentence, the larger the Perplexity.

A functional configuration of a language processing apparatus that performs language processing for generating a meaningless sentence for listening test using such a language processing method will be described. In FIG. 1, reference numeral 1 denotes a processing block that divides a set of input sentences (the five meaningful sentences described above) into three parts and replaces them to generate a meaningless sentence. The number of statements generated in this case is 1
4,400.

Reference numeral 2 denotes a morphological analysis block that divides the generated set of meaningless sentences into words. A storage block 4 stores a thesaurus. Reference numeral 6 denotes a word classification block for converting a word into a class code based on a thesaurus.

Numeral 8 denotes a storage block for storing a table of trigram probabilities (probability values of equation 3) between classes. 10 calculates Perplexity based on the class trigram for all the generated meaningless sentence pairs,
This is a processing block for selecting a set of meaningless sentences whose perplexity has increased.

FIG. 2 shows the contents of language processing executed in the language processing apparatus. The actual language processing will be described with reference to FIG. The execution contents in each block of FIG. 2 are as follows.

B2: Block for reading a set of input sentences.

B4: A block for generating a meaningless sentence by dividing each input sentence into three parts, fixing the first part, and exchanging the second and third parts. Five sentences

[0049]

## EQU6 ## b ₁₁ b ₁₂ b ₁₃ b ₂₁ b ₂₂ b ₂₃ b ₃₁ b ₃₂ b ₃₃ b ₄₁ b ₄₂ b ₄₃ b ₅₁ b ₅₂ b ₅₃ (b _ij is the j-th block of the i-th sentence) . Also, the function r is defined as {1, 2, 3, 4, 5} from {1,
A function to 2,3,4,5}, i ≠ j⇒r (i) ≠ r
(J) shall be satisfied. The function s is defined in the same manner. At this time, in this block,

[0050]

(Equation 7)

The sentence represented by is generated. Where r _i ,
s _j is an abbreviation for r (i) and s (i), respectively.

B6: Block for assigning 0 to a variable Max.

B8: A block for judging whether all the meaningless sentence sets generated in B4 have been selected.

B10: A block for selecting a not-yet-selected set from the set of meaningless sentences generated in B4.

B12: A subroutine for calculating the Perplexity of the set of meaningless sentences selected in B10. Details are shown in FIG. The input is a set of meaningless sentences, and the calculation result is returned to a variable out.

B14: A block for judging that the value of the variable out is larger than the variable Max.

B16: A block for determining that the value of the variable out is equal to the variable Max.

B18: A block for initializing an array for storing a set of output sentences and storing therein a currently selected set of meaningless sentences.

B20: Block for replacing the value of the variable Max with the value of the variable out.

B22: A block for adding a currently selected set of meaningless sentences to an array for storing sets of output sentences.

B24: A block for outputting a set of stored output statements.

FIG. 3 shows the contents of processing in the subroutine indicated by B12 in FIG. The processing block of FIG. 3 includes the following blocks.

B32: Block for assigning 0 to a variable out.

B34: A block for judging that all sentences have been selected when selecting a sentence from a set of input meaningless sentences.

B36: A block for selecting a sentence that has not been selected from the input sentence set.

B37: Morphological analysis block for dividing the selected sentence into words. As the morphological analysis method, any method may be used, but in the present embodiment, software “Juman” developed at Kyoto University is adopted.

B38: A block for decomposing a sentence into triads of words. Word statement w _1, w _2, ... If you are able to a concatenation of w _n, 3-tuple _{w 1 w 2 w 3, w} 2 w 3 w 4, ..., w n-2
Generate w _n-1 w _n .

B40: A block for classifying each word in the set of three words into a class and converting it into three sets of class codes.
Word three sets _{w 1 w 2 w 3, w} 2 w 3 w 4, ..., w n-2
w _{n- 1} w _n is _converted to a class triple set c ₁ c ₂ c ₃ , c ₂ c ₃ c
_4, ..., it is converted into _{c n-2 c n-1} c n.

B41: A block for reading out the trigram probability of Formula 3 for each triple generated in B40.

B42: From the trigram probability read out in B40,

[0071]

(Equation 8)

A block for calculating the Perplexity of a sentence.

B44: A block for substituting the sum of the value of the variable out and the value of Perplexity obtained in B42 for the variable out.

B46: Block for outputting the value of variable out.

(Second Embodiment) Next, an example will be described in detail below. In general, when a word trigram model is used in speech recognition, a triplet of words included in an uttered sentence did not exist at the time of learning, so that the probability value may not be estimated in some cases. In the example, for such a case, the probability value is estimated using the class trigram of Expression 2.

FIG. 4 shows a functional configuration of a language processing apparatus for executing the language processing of the example. In this example, it is assumed that there is some continuous speech recognition system, and this system outputs a plurality of recognition candidate sentences. In FIG.
Reference numeral 22 denotes a morphological analysis block that divides a plurality of sentences input as speech recognition candidates into words, and reference numeral 24 denotes a storage block that stores word trigram probabilities (Equation 1). 26 is a processing block for reading out the word chain probability corresponding to the input. 28 is a storage block for storing a thesaurus, and 30 is a word classification block for converting words to class codes based on the thesaurus.

Reference numeral 32 denotes a storage block for storing a table of inter-class trigram probabilities and class-word probabilities (probabilities that words appear in the class). 34 is the class trigram probability corresponding to the input sentence (probability of equation 2)
Is a processing block for calculating. Reference numeral 36 denotes an input sentence (recognition candidate) using the class trigram probability when the word trigram probability has not been obtained.
This is a processing block that calculates the Perplexity for, selects the one with the smallest value (the one with the highest probability value), and outputs it.

FIG. 5 shows the contents of language processing in Example 2. The actual language processing will be described with reference to FIG. The execution contents in each block of FIG. 5 are as follows.

B52: Block for reading a set of input sentences (a plurality of recognition candidates).

B54: A block for substituting the smallest possible value for the value of the variable Min.

B56: A block for determining whether or not all the input sentences have been selected.

B58: Block for selecting a sentence that has not been selected before.

B60: Perplexity for the selected sentence
Subroutine for calculating. The input is the selected sentence, and the Perplexity for it is returned using the variable out.

B62: Block for determining whether or not the variable out is smaller than the variable Min.

B64: Initialize the array for storing the output sentence, and store the currently selected sentence in the array.

B66: The value of the variable out is substituted for the variable Min.

B68: The sentence stored in the output sentence array is output.

FIG. 6 shows the processing contents of the subroutine B60. The processing block in FIG. 6 includes the following blocks.

B72: Block for assigning 0 to a variable out.

B73: A morphological analysis block for dividing the input sentence into words.

B74: Block for decomposing a sentence into triads of words. If the sentence is made up of the concatenation of the words w ₁ , w ₂ … w _n , the word triplet w ₁ w ₂ w ₃ , w ₂ w ₃ w ₄ ,.
to generate the _{n-2 w n-1 w} n.

B76: A block for judging whether a triple of all words has been selected.

B78: Block for selecting a word triplet that has not been selected.

B80: A block for determining whether or not a word trigram exists for the selected triplet.

B82: A block for adding the logarithmic value of the probability obtained from the word trigram model to the variable out.

B84: A word classification block for obtaining a class triple from the selected word triple.

B86: Word triad is w _m−2 w _m−1 w
_m , and the word class to which the word w _i belongs is c _i, and the chain probability for the triple of the class _cm −2 _{cm −1 cm} and c
A block for obtaining a logarithmic value of a chain probability of a word triplet w _m−2 w _m−1 w _m from the appearance probability of the word w _m under the condition where _m is given, according to Equation ₂ .

B88: The probability value obtained in B86 is replaced by a variable ou
Block to add to t.

B90: Block for outputting variable out.

In the present embodiment, when creating a word class, the last one digit of the code number of the classification vocabulary table is truncated and created. It is also applicable to the method of creating. Furthermore, although the classification vocabulary table of the municipal language institute was used as the thesaurus, it goes without saying that the present invention can be applied to the case where a thesaurus other than the classification vocabulary table is used. Also, n-
Although illustrated only in the case of n = 3 (trigram) in the gram model, it is needless to say that the present invention can be applied to other values of n.

Experimental results for an example of generating a meaningless sentence using the above-described five sentences are shown below. The result of arranging the generated set of meaningless sentences in descending order of Perplexity (that is, in order of decreasing occurrence probability) is as follows: Perplexity: 260.36 A motorcycle is smoked. The building has not been installed since this morning. There are many pigeons in the drawer. The key hurts in my head. High teeth stopped.

Perplexity: 260.36 Tobacco stopped on motorcycles. From this morning the building
I'm sorry. There are many pigeons in the drawer. The key hurts in my head. High teeth.

Perplexity: 260.27 Motorcycles are smoking. The building hurts from this morning. There are many pigeons in the drawer. To head
I can't find the key. High teeth stopped.

Perplexity: 260.27 Tobacco stopped on motorcycles. From this morning the building
painful. There are many pigeons in the drawer. A key in my head
I'm sorry. High teeth.

Perplexity: 258.92 There are many pigeons on motorcycles. The building has not been installed since this morning. There is a cigarette in the drawer. The key hurts in my head. High teeth stopped.

Perplexity: 258.92 There are many pigeons on motorcycles. The building has not been installed since this morning. Tobacco stopped in the drawer. The key hurts in my head. High teeth.

Perplexity: 258.83 There are many pigeons on motorcycles. The building hurts from this morning. There is a cigarette in the drawer. To head
I can't find the key. High teeth stopped.

Perplexity: 258.83 There are many pigeons on motorcycles. The building hurts from this morning. Tobacco stopped in the drawer. A key in my head
I'm sorry. High teeth.

On the other hand, for comparison, similar processing was performed using the word trigram (Equation 1) in the block B40 in FIG. As a result, 16 sets of sentences with the maximum Perplexity (minimum probability value) were obtained. Some of these are listed below: Perplexity: 718.67 No tobacco on motorcycles. From this morning my teeth
painful. The pigeon stopped in the drawer. There is a building on my head. There are many high keys.

Perplexity: 718.67 No tobacco is available on motorcycles. From this morning my teeth
It depends. The pigeon stopped in the drawer. The building hurts on my head. There are many high keys.

[0111] Perplexity: 718.67 No tobacco is available on motorcycles. From this morning my teeth
It depends. The pigeon stopped in the drawer. There are many buildings in my head. The high key hurts.

Perplexity: 718.67 No tobacco is available on motorcycles. I have a lot of teeth from this morning. The pigeon stopped in the drawer. There is a building on my head. The high key hurts.

Perplexity: 718.67 Tobacco stopped on the motorcycle. My teeth have started this morning. There are no pigeons in the drawer. The building hurts on my head. There are many high keys.

Perplexity: 718.67 Tobacco stopped on motorcycles. My teeth have started this morning. There are no pigeons in the drawer. There are many buildings in my head. The high key hurts.

Perplexity: 718.67 Tobacco stopped on motorcycles. I have a lot of teeth from this morning. There are no pigeons in the drawer. There is a building on my head. The high key hurts.

Comparing these experimental results, when the language processing method of the present embodiment is used, the meaningless sentence is stably constructed, whereas when the word trigram is used, the original meaningless sentence is obtained. Result is also included. This shows that the present invention has a sufficient effect.

Next, in order to examine the effect of the example, the present invention was compared with the case using back-off smoothing by calculating the Perplexity using the transcript of the news actually broadcast. The evaluation data used is a text transcribed from the occurrence of an anchor in a certain news program on June 4, 1996, and does not include a sentence duplicated with the learning database. As a result, the Perplexity when the back-off smoothing was used was 62.38, whereas the Perplexity when the language processing method of the present embodiment was used was 5
7.97 was obtained, and the effect of the present invention was confirmed.

In this experiment, first, a word trigram model, a class n-gram model were used as a word n-gram model.
A class trigram model was adopted as a model, and only the two were compared by perplexity. For word trigrams, we used Katz's back-off smoothing. Table 2 shows the results. From Table 2, the word trigram showed lower perplexity than the class trigram.

[0119]

[Table 2]

In order to investigate the reason, the result of the evaluation data was examined in detail. As an example of reducing the perplexity of the word trigram model, an example in which the trigram has a high value will be described.

[0121]

P (Practice | Pacific Rim, Joint) = 1 P (Council | Central, Education) = 1 In these examples, the probability of the trigram is 1. In other words, in the training data,
When a two-word chain of "joint" appears, it means that "practice" always follows. The same applies to "center", "education" and "council". Since such a chain of three words also appeared in the evaluation data, it is considered that the value of the word perplexity was lowered.

On the other hand, in the case of the class trigram model, even if “learning” is the only word following “ring pacific” and “joint” in the learning data, each is replaced with a class. Therefore, the probability of the class linkage is not always 1. Further, since the probability of this class chain is multiplied by the appearance probability of “exercise” in the class (1.305) to which “exercise” belongs, the probability value of the class trigram model is further reduced. For these reasons, it is considered that the perplexity of the class trigram model is larger than the case where the word trigram model is used.

From the above examination, the following conclusions can be obtained: (1) The word trigram model has a higher adaptability to learning data than the class trigram model. Therefore, when the statistical properties of the evaluation data and the learning data are similar, high expression ability is exhibited.

(2) The class trigram model has the same effect as smoothing because it utilizes the statistical properties of each class. Therefore, as shown in Equation 9, the probability value is 1
However, it is possible to estimate the probability value of a word chain that does not exist in the learning data.

From the above (2), a method is conceivable in which the class trigram is used as a smoothing model for the word trigram. Therefore, when a triple of words not included in the word trigram model appears, a composite model for estimating a probability value by a class trigram model without using back-off smoothing is constructed, and this model is used. The perplexity for the evaluation data was determined. As a result, for perplexity 62.38 in the case of word trigram + back-off smoothing,
A result of 57.97 was obtained, making it possible to further reduce perplexity.

[0126]

As described above, according to the first and eighth aspects of the present invention, words are semantically classified according to the thesaurus, so that the number of types of classification is larger than that of parts of speech, Also have a high degree of relevance, thereby increasing the accuracy of the linkage probability.

According to the second and ninth aspects of the present invention, the word itself is not used when acquiring the chain probability between words, but the classification code of the classification including the word is used.
The number of combinations of classification codes for storing the chain probabilities of the learned classification codes is significantly reduced as compared with the combinations of words. Thus, the memory capacity of the language processing device can be made smaller than before.

According to the third and tenth aspects of the present invention, by using an auxiliary word that is not described in the thesaurus as a classification code, the auxiliary word can be handled.

According to the fourth and eleventh aspects of the present invention, a large number of words can be handled by classifying parts of speech for independent words not described in the thesaurus classification.

According to the fifth and twelfth aspects of the present invention, a meaningless sentence used in the psychoacoustic experiment can be created by selecting a word string with the smallest chain probability, and the created sentence is also arbitrary. Contains no elements.

According to the sixth and thirteenth aspects of the present invention, the word string having the largest chain probability is determined as the optimum candidate among the speech recognition candidates. And the speech recognition accuracy is improved.

According to the seventh and fourteenth aspects of the present invention, even when the chain probability between words is not given by learning or the like, the chain probability between words can be estimated, so that the number of words to be handled can be increased. it can.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a functional configuration of a language processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure of the language processing device according to the first embodiment of the present invention.

FIG. 3 is a flowchart illustrating a processing procedure of the language processing apparatus according to the first embodiment of the present invention.

FIG. 4 is a block diagram illustrating a functional configuration of a language processing device according to a second embodiment of the present invention.

FIG. 5 is a flowchart illustrating a processing procedure of a language processing device according to a second embodiment of the present invention.

FIG. 6 is a flowchart illustrating a processing procedure of a language processing apparatus according to a second embodiment of the present invention.

[Explanation of symbols]

Reference Signs List 1 nonsense sentence generation block 2 morphological analysis block 4 thesaurus table storage block 6 word classification block 8 processing block for storing a table of trigram probabilities between classes 10 processing block for selecting a set of nonsense sentences 22 morphological analysis block 24 words Trigram probability storage block 26 Processing block for reading word chain probabilities 28 Thesaurus storage block 30 Word classification block 32 Storage block for storing a table of trigram probabilities and class-word probabilities between classes 34 Processing block for calculating class trigrams 36 Processing block for selecting recognition candidates using class trigram probability

Claims (14)

[Claims] 1. Classifying a plurality of words into a plurality of classes in which one class is composed of a plurality of sets of words, acquiring a linkage probability indicating a degree of linkage between the classified classes,
In a language processing apparatus for selecting a class and / or a word string based on the acquired chain probability, a storage means for storing a thesaurus classification; and a classification means for classifying the plurality of words according to the thesaurus classification A language processing device, comprising: 2. The language processing apparatus according to claim 1, wherein the classifying unit is configured to classify the plurality of words.
A language processing apparatus for converting a word into a classification code, and calculating the word string in the form of a classification code string when calculating the linkage probability. 3. The language processing apparatus according to claim 2, wherein said classifying means uses, as a classification code, the attached word itself for an attached word attached to said word. 4. The language processing apparatus according to claim 2, wherein the classification unit performs classification for each part of speech when the word is an independent word that is not described in the thesaurus classification. Language processing device. 5. The language processing apparatus according to claim 1, further comprising a selection unit that selects a word string having the smallest chain probability as a meaningless sentence used in a hearing experiment. apparatus. 6. The language processing apparatus according to claim 1, further comprising a selection unit that selects a word string having the largest chain probability as an optimal candidate among speech recognition candidates. apparatus. 7. The language processing apparatus according to claim 1, wherein a probability of appearance of a word in a class and a probability of linkage between classes are obtained in association with calculation of a chain probability between words. If the chain probability between them cannot be obtained directly,
A language processing apparatus for estimating a chain probability between the words based on an appearance probability of the words and a chain probability between the classes. 8. Classifying a plurality of words into a plurality of classes in which one class is composed of a plurality of words, acquiring a linkage probability indicating a degree of linkage between the classified classes,
A language processing method for selecting a class and / or a word string based on the acquired chain probability, wherein a classification of a thesaurus is stored, and the plurality of words are classified according to the classification of the thesaurus. Language processing method. 9. The language processing method according to claim 8, wherein when the plurality of words are classified, the words are converted into a classification code, and when the chain probability is calculated, the word string is calculated in the form of a classification code string. A language processing method. 10. The language processing method according to claim 9, wherein, for an attached word attached to the word, the attached word itself is used as a classification code. 11. The language processing method according to claim 9, wherein, if the word is an independent word that is not described in the thesaurus classification, classification is performed for each part of speech. 12. The language processing method according to claim 8, wherein the word sequence with the smallest chain probability is selected as a meaningless sentence used in a hearing experiment. 13. The language processing method according to claim 8, wherein the word sequence having the largest chain probability is selected as an optimal candidate among the speech recognition candidates. 14. The language processing method according to claim 8, wherein the probability of appearance of a word in a class and the probability of linkage between classes are obtained in association with calculation of a chain probability between words. If the chain probability between them cannot be obtained directly,
A language processing method comprising: estimating a chain probability between the words based on an appearance probability of the word and a chain probability between the classes.