JP5766152B2 - Language model generation apparatus, method and program - Google Patents

Description

  The present invention relates to a technique for generating a language model from a text corpus.

  Currently, probabilistic language models (hereinafter also simply referred to as “language models”) are used in various fields such as speech recognition and automatic translation. A language model is a model that gives the probability of occurrence to word strings and character strings. The n-gram model is the most common language model (see Non-Patent Document 1). The n-gram model is a model based on the assumption that the word occurrence probability depends only on the most recent (n-1) words. For example, considering the sentence “I am an apple, ____”, it is assumed that the words that are underlined are “eating”, “buying”, “gazing”, and the like. This is inferred from the sequence of the words “apple” and “wo” appearing before the underlined portion. As described above, it is effective to use the appearance information of several words just before the word occurrence probability at a certain point in time.

  In general, a case where only the information of the immediately preceding one word is used is called a bigram, and a case where the information of the immediately preceding two words is used is called a trigram. For example, in the above example, when considering a word that falls underlined, only “wo” is considered in the bigram, and “apple” and “ho” are considered in the trigram.

For example, in the above example, the conditional probability P that “eat” enters the underlined portion is calculated as follows in the bigram model and the trigram model, assuming that the appearance frequency of the word string W is C (W). .
bigram: P (eat |) = C (eat-eat) / C ()
trigram: P (eat | apple-) = C (apple-eat) / C (apple-)
Usually, in the field of speech recognition, n = 2 (bigram) and n = 3 (trigram) are often used.

  In general, trigram has higher estimation accuracy than bigram. For example, the problem of estimating the word following the word strings “apple” and “to” is easier than the problem of estimating the word following the word “o”. Therefore, an ideal language model is a case where a conditional probability (hereinafter also referred to as “trigram probability”) calculated by any trigram model is in a state equal to an actual appearance distribution.

  It is desirable that the conditional probability (hereinafter also referred to as “bigram probability”) and trigram probability calculated by such a bigram model is usually learned from a large amount of learning corpus. A corpus is a database composed of text data generated based on a natural language. The larger the size of the learning corpus, the more n-gram patterns (word string patterns made up of n words) can be learned, and the conditional probability calculated by the n-gram model ( (Hereinafter also referred to as “n-gram probability”) is a statistically high value. That is, the accuracy of the language model is increased. On the other hand, when the learning corpus size is small, a sufficient n-gram pattern cannot be covered, and the n-gram probability is statistically low in reliability. That is, the accuracy of the language model is low. Thus, in order to improve the accuracy of the language model, a large amount of learning corpus is required.

  The learning corpus is preferably the same as the actual task. For example, when a language model is used for speech recognition, it is desirable that the learning corpus has an appearance frequency distribution of words equivalent to a task that is a speech recognition target. For example, the appearance tendency of words used in baseball broadcasts is different from the appearance tendency of words used in telephone receptions such as call centers. Therefore, when trying to create subtitles for a baseball broadcast using speech recognition, it is better to use the text model that transcribes the content of the baseball broadcast as a learning corpus and the text that transcribes the content of the telephone response. The recognition accuracy is higher than when a language model generated as a learning corpus is used.

Kenji Kita, “Language and Computation 4 Stochastic Language Model”, 1999, The University of Tokyo Press, p57-62

  However, as described above, in order to generate a highly accurate language model, a large amount of learning corpus is required, and when only a small amount of learning corpus can be prepared, a highly accurate language model cannot be generated. In particular, if a learning corpus for a specific task is prepared, a large number of learning corpora cannot be prepared in many cases.

  Further, when generating a language model used in speech recognition, the recognition accuracy is higher when text data written from speech is used as a learning corpus. At this time, in order to transcribe speech and create a large amount of learning corpus, it is necessary to manually transcribe a large amount of speech, which requires a large cost (time and labor costs). Furthermore, if a large amount of learning corpus is prepared for each task, the cost becomes even higher. In order to reduce this cost, if a language model is generated from a small amount of learning corpus, its accuracy is lowered.

  An object of the present invention is to provide a technique for generating a language model with higher accuracy than a conventional technique from a small amount of text corpus.

  In order to solve the above-described problem, according to the first aspect of the present invention, the language model generation device uses the original text that is divided into morpheme units and added with the dependency relationship of clauses. N-gram using a pseudo-text generation unit that generates a pseudo-text by rearranging a plurality of clauses having the same number, an appearance frequency of an n-gram pattern in the original text, and an appearance frequency of the n-gram pattern in the pseudo-text A language model generation unit for obtaining a probability and generating a language model.

  In order to solve the above-mentioned problem, according to the second aspect of the present invention, the language model generation method uses the original text that is divided into morpheme units and added with the dependency relation of clauses. A pseudo-text generation step for rearranging a plurality of clauses having the same number to generate pseudo-text, and an n-gram pattern using the appearance frequency of the n-gram pattern in the original text and the appearance frequency of the n-gram pattern in the pseudo text A language model generation step for determining a probability and generating a language model.

  According to the present invention, by increasing the n-gram pattern acquired from one sentence, there is an effect that a language model with higher accuracy can be generated from a small amount of text corpus than in the conventional technology.

FIG. 1A is a diagram for explaining a dependency relation between clauses, and FIG. 1B is a diagram for explaining a syntax analysis result. The functional block diagram of the language model production | generation apparatus which concerns on 1st embodiment. The figure which shows the processing flow of the language model production | generation apparatus which concerns on 1st embodiment. The figure for demonstrating the method to rearrange the some clause whose dependency is the same clause. The functional block diagram of the language model production | generation apparatus which concerns on 2nd embodiment. The figure which shows the processing flow of the language model production | generation apparatus which concerns on 2nd embodiment. The functional block diagram of the pseudo text selection part which concerns on the 1st determination method of 2nd embodiment. The figure which shows the processing flow of the pseudo text selection part which concerns on the 1st determination method of 2nd embodiment. The functional block diagram of the pseudo text selection part which concerns on the 2nd determination method of 2nd embodiment. The figure which shows the processing flow of the pseudo text selection part which concerns on the 2nd determination method of 2nd embodiment. The functional block diagram of the language model production | generation apparatus which concerns on 3rd embodiment. The figure which shows the processing flow of the language model production | generation apparatus which concerns on 3rd embodiment.

<Points of first embodiment>
The following seven trigram patterns are learned from the sentence “I eat that apple with my friend today (I / ha / that / apple / to / today / friend / and / eat)”. However, the results in parentheses are divided into morpheme units.
1. I-that-2. Ha-that-apple3. That apple-4. Apple-today -5. -Today-Friends 6. 6. Today-with friends- In this embodiment to eat with friends, it is desired to increase the n-gram pattern (for example, trigram pattern) obtained from a certain sentence.

  Therefore, the present embodiment focuses on the “word order fluctuation” characteristic of Japanese. Japanese is a language that is prone to change in word order, especially in spoken language. For example, the sentence "I eat that apple with my friend today" is wrong as a Japanese line even if I say "I eat that apple with my friend today" or "I eat that apple with my friend today" Not. Thus, it is difficult to uniquely determine the word order in Japanese. And it is difficult to cover various word order changes with a small amount of learning corpus. In view of this, for each text in a certain learning corpus, text in which the word order is changed is created and used as a learning corpus to increase the number of n-gram patterns to be learned. An original learning corpus is called an original text corpus, and text data in the original text corpus is called an original text. Text in which the word order of the original text is changed is called pseudo text, and a corpus composed of pseudo text is called a pseudo corpus. The original text corpus and the pseudo corpus are used together as a learning corpus.

For example, “I eat that apple with my friend today” is not unnatural as a Japanese sequence even if it is expressed in the following word order:
Original text: Pseudo-text I eat that apple with friends today (1): Pseudo-text I eat that apple with friends today (2): Pseudo-text I eat that apple with friends today (3): I Pseudo-text to eat that apple with a friend today (4): Pseudo-text to eat that apple with a friend today (5): Pseudo-text to eat that apple with a friend today (6): Today I'm a friend with that apple Eat with ...
By rearranging as described above, it becomes possible to learn trigram patterns that were not included in the original sentence, such as “today-that-apple”, “friends-to-today”, and “today-me-ha”. . For example, the following seven trigram patterns (1) 1 to (1) 7 are learned from the pseudo text (1) “I eat that apple with my friend today”. (1) 1 to (1) 5 and (1) 7 are trigram patterns newly acquired by the pseudo text (1).
(1) 1. Today -I- (1) 2. My friend (1) Ha-friends-(1) 4. Friends-and-(1) 5. And-that-apple (1) 6. That apple-(1) 7. By eating apples in this way, pseudo-text is generated from the original text by rearrangement, so that a new n-gram pattern that cannot be obtained from the original text can be extracted.

In the present embodiment, “dependency relation” is used to realize word order fluctuation. “Dependency relationship” in Japanese means that a phrase and a phrase are related to each other with a meaningful connection (modifier and modifier). “Sentence” is a group of meanings that is the smallest when a sentence is divided into small pieces. In general, a phrase is composed of “independent words” such as nouns and verbs and “junctions”. “Suffix” may be omitted or omitted. For example, “I eat that apple with my friend today” can be broken into the following phrases:
Original text: I will eat that apple with friends today: I will eat / that / apples / today / with friends / Eat in such a delimited clause, each clause is a dependency relationship as shown in FIG. 1A Can be extracted. In the case of the example in FIG. 1A, a total of five dependency relationships are extracted: “I eat →” “that → apple”, “apple → eat”, “today → eat”, and “friend → eat”. Between the clauses in the dependency relationship, a direct connection relationship is established from what is modified to what is modified. In addition, clauses having the same dependency relationship are independent of each other.

  “I” (that) apples, “today” and “with friends” are involved in “eating”. Rearranging the four phrases related to “eat” is not an error in the Japanese word order. By rearranging as described above, it is possible to learn trigram patterns such as “Today-That-Apple”, “Friends-To-Today”, “Today-I-Ha” etc. that were not included in the original text. Become. By using the dependency relationship in this manner, more natural n-gram patterns can be extracted from one sentence.

  Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted.

<First embodiment>
FIG. 2 is a functional block diagram of the language model generation apparatus 100, and FIG. 3 shows a processing flow thereof.

  The language model generation apparatus 100 includes a morphological analysis unit 110, a syntax analysis unit 120, a pseudo text generation unit 130, and a language model generation unit 140.

Language model generator 100 receives the T original text tex _t in the original text corpus to generate a language model using the original text tex _t, and outputs. However, t = 1, 2,..., T. Details of each part will be described below. In the present embodiment, the original text corpus only needs to include text data composed of the original text, and part-of-speech information is not necessarily required.

<Morphological analyzer 110>
• Input: Original text tex _t
- Output: the result of morphological analysis (the original text has been word-separated into morphemes) mor _t
- processing content: The morphological analysis of the original text tex _t (s110), by dividing the original text into morphemes and outputs the (word-separated by the original text into morphemes) mor _t morphological analysis result. A morpheme is the smallest unit that has linguistic significance. Conventional technology is used as the morphological analysis technology. For example, if "I am that apple a day to eat with your friends" to morphological analysis of the original casing text, as shown in the following, the word is "/" in delimited format of the morphological analysis result mor _t is obtained.
⇒I / Ha / Ano / Apple / Today / Friends / To / Eat

<Syntax analyzer 120>
• Input: morphological analysis results (leaving a space between words is the original text into morphemes) mor _t
Output: Syntax analysis result (information indicating morphological analysis result and clause dependency) syn _t
- processing content: The morphological analysis result parsing mor _t (s120), by dividing the morphological analysis result mor _t in clause analyzes the dependency relationships between the plurality of divided clauses, syntax analysis result (morphological analysis result clause dependency information indicating a relationship) to the syn _t. In the present embodiment, the syntax analysis means analyzing the dependency relation of clauses. Conventional techniques are used as the parsing technique. For example, "I am / is / that / apple / a / today / friend / and / eat" syntax analysis result syn _t like that and do the syntax analysis on the morphological analysis result FIG. 1B is obtained. In this specification, the dependency relationship as shown in FIG. 1B is referred to as “I / I (6) That (3) Apple / (6) Today (6) Friends / (6) Eat” for convenience. The number in parentheses means the number of the clause that the previous clause is related to. For example, the first phrase “I / ha” is related to the sixth phrase “eat”.

<Pseudo Text Generation Unit 130>
Input: Syntax analysis result (information indicating morphological analysis result and clause dependency) syn _t
-Output: pseudo text tex _{t, u}
Processing content: Using the syntax analysis result syn _t , the respective clauses are rearranged to generate U _t pieces of pseudo text tex _{t, u} (s130). However, u = 1, 2,..., U _t . The rearrangement is performed by rearranging a plurality of phrases having the same dependency destination. For example, if you receive “I / I (6) That (3) Apple / (6) Today (6) Friends / (6) Eat”), the 6th sentence “Eat” will be the first The four clauses of the phrase “I / ha”, the third clause “(that) / apple”, the fourth clause “today”, and the fifth clause “friend / to” are rearranged. By rearranging these four clauses according to the permutation combination, pseudo text tex _{t, u} is generated. Therefore, (4! -1 = 4 × 3 × 2 × 1-1 = 23) pseudo-text tex _{t, u} is generated (see FIG. 4). It should be noted that, "- 1" corresponds to the original text tex _t. If there is no clause having the same dependency destination for a certain parsing result syn _t , U _t = 0, and no pseudo text tex _{t, u} is generated.

<Language model generation unit 140>
-Input: Original text tex _t , pseudo text tex _{t, u}
・ Output: Language model (n-gram model)
Processing contents: Appearance frequency Count _G of n-gram patterns in T original texts tex _t and appearance frequency Count _S of n-gram patterns in (U ₁ + U ₂ +... + U _T ) pseudo texts tex _{t, u} The n-gram probability is obtained from the above and a language model is generated (s140). Incidentally, when determining the n-gram probability, frequency of occurrence of n-gram patterns obtained from the T original text tex _t Count _G and _{(U 1 + U 2 + ...} + U T) pieces of pseudo-text _{tex t,} from _u You may weight to the appearance frequency Count _S of the n-gram pattern obtained. For example, the bigram probability obtained by weighted mixing with the weight W is calculated by the following equation.

The trigram probability obtained by performing weighted mixing with the weight W is calculated by the following equation.

However, the weight W is set to a value larger than 0, and if the weight W is 1, it means that the original text tex _t and the pseudo text tex _{t, u} are totaled with the same weight. Normally, it is considered that the original text tex _t is more likely in word order than the pseudo text tex _{t, u} , so it is desirable to set W to 1 or less. For example, the weight W is determined by a value that generates a language model that maximizes the recognition accuracy of the development set.

<Effect>
With such a configuration, the n-gram pattern acquired from one sentence (original text tex _t ) can be increased, and a highly accurate language model can be generated from a small amount of text corpus as compared with the prior art.

<Modification>
Language model generator 100, without providing the morphological analysis unit 110 and the syntax analyzing unit 120 may be input, for example, other previously obtained morphological analysis result by the apparatus mor _t and syntax analysis result syn _t.

  Further, when generating an n-gram model, a known method called smoothing or smoothing may be used (see Non-Patent Document 1).

<Second embodiment>
Only parts different from the first embodiment will be described.

  When there is an error in the syntax analysis in the syntax analysis unit 120, there is a possibility that a pseudo-text having an originally incorrect sentence type is generated due to the error. As a result, the performance of the language model generated later may be deteriorated. Therefore, in the second embodiment, a processing unit that determines whether or not the pseudo text is likely to be in word order is added.

  FIG. 5 is a functional block diagram of the language model generation apparatus 200, and FIG.

  The language model generation apparatus 200 includes a morphological analysis unit 210, a syntax analysis unit 120, a pseudo text generation unit 130, a language model generation unit 140, and further includes a pseudo text selection unit 250.

<Morphological analyzer 210>
• Input: Original text tex _t
-Output: Morphological analysis result (original text divided into morpheme units and with part-of-speech information added) mor ' _t
And processing the contents: a and morphological analysis original text tex _t (s210), to divide the original text into morphemes, by applying a part of speech for each morpheme obtained by dividing, is leaving a space between words in the morphological analysis result (morpheme units, is part of speech information and it outputs the added original text) mor _'t. Conventional technology is used as the morphological analysis technology. For example, "I have that apple today to eat with my friends" and morphological analysis of the original casing text, as follows, words are separated by a "/", morphological analysis result mor _'t of the format that has been added to part of speech can be obtained .
⇒ I (noun: pronoun) / ha (conjunctive particle) / that (combined particle) / ringo (noun) / a (case particle: conjunctive) / today (noun: date: conjunctive) / friend (noun) / to (case particle) : Continuous use) / eat (verb)

<Pseudo-text selection unit 250>
Input: (part of speech information is added because it is generated using a parse analysis result syn ' _t composed of a morphological analysis result mor' _t to which part of speech information is added and information indicating a dependency relationship between clauses) Pseudo-text tex ' _{t, u} , morpheme analysis result (original text divided into morpheme units and with part-of-speech information added) mor' _t
-Output: Selection pseudo text tex ' _{t, y}
Processing content: using the word sequence of the original text tex _t , determine whether the word sequence of the pseudo text tex ′ _{t, u} is correct _, and select the pseudo text tex ′ _{t, u} determined to be correct (S250), it is output to the language model generation unit 140 and used for learning the language model. If it is determined that it is not correct, the pseudo text tex ′ _{t, u} is not selected and is not used for language model learning. In the present embodiment, the part-of-speech word order is used to determine whether the word sequence is correct, and the part-of-speech word order of the original text tex _t is compared with the part-of-speech word order of the pseudo-text tex ′ _{t, u.} Then, the pseudo-text tex ′ _{t, u} having the word order of a certain part of speech is selected. Two methods for determining whether or not the word order of the part of speech of the pseudo text tex ′ _{t, u} is correct will be described below.

(1) First determination method The first determination method will be described with reference to FIGS. 7 and 8. The pseudo text selection unit 250 includes a first part-of-speech information acquisition unit 251, an appearance part-of-speech string collection storage unit 253, a second part-of-speech information acquisition unit 255, and a determination unit 257. First, the first part-of-speech information acquisition unit 251, a set of morphological analysis result mor _'t take out part of speech information added to the original text tex _t from (s251), the part of speech of the T of the original text tex _t word order, appearance The part-of-speech string set is stored in the appearance part-of-speech string set storage unit 253 (s253). Next, the second part of speech information acquisition unit 255 outputs the pseudo text tex _'t, the part of speech information added to _u, pseudo text tex' _t, removed the part of speech of the word order of _u (S255), the determination unit 257 . The determination unit 257 receives the part-of-speech word order of the pseudo-text tex ′ _{t, u} and determines whether or not a similar part-of-speech word order exists in the appearance part-of-speech string set storage unit 253 (s257). If it exists, it is determined that the word order of the part of speech is probable, the pseudo text tex ' _{t, u} corresponding to the word order of the part of speech is selected (s258), and the language model is selected as the selected pseudo text tex' _{t, y.} The data is output to the generation unit 140. However, y = 1, 2,..., Y _t , and Y _t is a selection pseudo text tex ′ _{t, y} selected from U _t pseudo texts tex ′ _{t, u} obtained from a certain original text tex _t . It is a number. If it does not exist, it is determined that the pseudo text tex ′ _{t, u} is an originally incorrect sentence type and is not selected.

Note that the word order of the part of speech of the pseudo text tex't _{, u} and the word order of the part of speech included in the appearing part of speech sequence set in the appearance part of speech sequence set storage unit 253 are not necessarily the same, and a predetermined ratio ( For example, the pseudo-text tex ′ _{t, u} may be selected when 90%) or more are the same. In other words, if the word order of the part of speech of the pseudo text tex ′ _{t, u} and the word order of any part of speech included in the appearance part of speech sequence set match a predetermined ratio or more, the pseudo text tex ′ _{t, u} is selected. May be. When the word order is the same, the selection of the pseudo text tex ′ _{t, u} is determined experimentally so that the recognition accuracy is improved. For example, when the part-of-speech word order of the pseudo-text is composed of ten part-of-speech word orders, the part-of-speech sequence set is extracted from the part-of-speech sequence set and the part-of-speech word order is compared. If there is a match, select the pseudo-text. Note that the matching ratio may be obtained by other methods.

  If the corpus size of the original text corpus is not sufficiently large, select that the part-of-speech word order of the pseudo-text is the same part-of-speech part order in the appearance part-of-speech sequence set (in other words, the predetermined percentage is 100%) In this condition, since there are few types of part-of-speech word order included in the appearance part-of-speech string set, many pseudo-texts are not selected. Then, the corpus sizes of the pseudo corpus and the learning corpus are reduced, and as a result, the accuracy of the language model may be lowered. In such a case, by reducing the matching ratio, the corpus sizes of the pseudo corpus and the learning corpus can be increased, and as a result, the accuracy of the language model can be improved.

(2) Second determination method The second determination method will be described with reference to FIGS. 9 and 10. The pseudo-text selection unit 250 includes a part-of-speech n-gram probability calculation unit 258 and a part-of-speech n- in addition to the first part-of-speech information acquisition unit 251, the appearance part-of-speech sequence storage unit 253, the second part-of-speech information acquisition unit 255, and the determination unit 257. A gram probability storage unit 259 is further included. The processes in the first part-of-speech information acquisition unit 251, the appearance part-of-speech string set storage unit 253, and the second part-of-speech information acquisition unit 255 are the same as those in the first determination method.

The part-of-speech n-gram probability calculation unit 258 extracts the appearance part-of-speech sequence set in the appearance part-of-speech sequence storage unit 253 and calculates the part-of-speech n-gram probability for the part-of-speech n-gram pattern included in the appearance part-of-speech sequence set ( s258) and stored in the part-of-speech n-gram probability storage unit 259 (s259, where FIG. 10 illustrates the case where the part-of-speech trigram probability is used as the part-of-speech n-gram probability). For example, if the appearance frequency of the part-of-speech string W in the appearance part-of-speech string set is expressed as C (W), the part-of-speech bigram probability and the part-of-speech trigram probability are calculated as follows. In the following expression, A, B, and C each represent a part of speech, “−” represents a connection of parts of speech, for example, B-A represents that part of speech A appears after part of speech B.
Part of speech bigram probability: P (A | B) = C (BA) / C (B)
Part of speech trigram probability: P (A | BC) = C (BCA) / C (BC)

Determination unit 257, pseudo-text tex _'t, receives the part of speech of the word order of _u, pseudo text tex' _t, the part-of-speech n-gram probability corresponding to the part-of-speech n-gram patterns obtained from the part of speech of the word order of _u-speech n- Extracted from the gram probability storage unit 259 (s257a). For example, as the word order of the part of speech of the pseudo-text tex't _{, u} , (conjunctive) (noun: pronoun) (conjunctive particle) (noun) (case particle: joint use) (noun) (case particle: joint use) (noun: date: When the (continuous) (verb) is received, the part-of-speech trigram probabilities corresponding to the following seven part-of-speech trigram patterns are extracted from the part-of-speech n-gram probability storage unit 259.
1. (Conjunctive)-(noun: pronoun)-(continuous particle)
2. (Noun: pronoun)-(continuous particle)-(noun)
3. (Consecutive particle)-(noun)-(case particle: consecutive)
4). (Noun)-(case particle: continuous use)-(noun)
5. (Case particle: continuous use)-(noun)-(case particle: continuous use)
6). (Noun)-(case particle: continuous use)-(noun: date: continuous use)
7). (Case particle: continuous)-(noun: date: continuous)-(verb)
The extracted part-of-speech n-gram probability is compared with a predetermined threshold value (s257b), and if it is equal to or greater than the threshold value, the word order of the part-of-speech is determined to be probable, and the pseudo-text tex ′ _{t, u} corresponding to the word order of the part-of-speech Is selected (s258), and is output to the language model generation unit 140 as selection pseudo text tex't _{, y} . If it is less than the threshold, it is determined that the pseudo text tex ′ _{t, u} is an originally incorrect sentence type and is not selected.

  The following method can be considered as a method of comparing with the threshold.

    (I) The average value of the extracted part-of-speech n-gram probabilities is obtained, and the average value is compared with a threshold value. If the average value is greater than or equal to the threshold, it is determined that the word order of the part of speech is likely.

(Ii) Each of the extracted part-of-speech n-gram probabilities is compared with a threshold, and if the M _{t, u} part-of-speech n-gram probabilities are equal to or greater than the threshold, it is determined that the word order of the part of speech is likely. However, if the number of part-of-speech n-gram patterns included in the pseudo text tex ′ _{t, u} is N _{t, u} , then M _{t, u} ≦ [VN _{t, u} ], and 0 <V ≦ 1,・] Represents the following maximum integer. V is determined experimentally so that the recognition accuracy is improved.

<Language model generation unit 140>
The language model generation unit 140 receives, as an input, not all (U ₁ + U ₂ +... + U _T ) pseudo texts tex ′ _{t, u} generated by the pseudo text generation unit 130, but the pseudo text selection unit 250 among them. A language model is generated using only the (Y ₁ + Y ₂ +... + Y _T ) selection pseudo-texts tex ′ _{t, y} selected in (S140). The method for generating the language model is the same as in the first embodiment.

<Effect>
With such a configuration, the same effect as that of the first embodiment can be obtained. Furthermore, it is possible to prevent a language model from being generated using pseudo-text tex ′ _{t, u} having an originally incorrect sentence type, and to prevent performance degradation of the language model.

<Third embodiment>
Only parts different from the second embodiment will be described.

In the third embodiment, the language model generation unit 140 changes the weight W (see formula (1) and formula (2)) between the original text tex _t and the pseudo text tex _{t, u} for each pseudo text tex _{t, u} . In the generated pseudo-text tex _{t, u} , from the viewpoint of “probability”, it is a word order that is likely to give the same frequency as the T original text tex _t , or a word order that is not mistaken but is not often used. It can also be considered. Therefore, in the third embodiment, a process of calculating the weight W for each pseudo text tex _{t, u} is added.

  FIG. 11 is a functional block diagram of the language model generation apparatus 300, and FIG. 12 shows its processing flow.

  The language model generation apparatus 300 includes a morphological analysis unit 210, a syntax analysis unit 120, a pseudo text generation unit 130, a language model generation unit 140, a pseudo text selection unit 250, and further includes a pseudo text weight calculation unit 370.

<Pseudo Text Weight Calculation Unit 370>
Input: selection pseudo-text tex ′ _{t, y} (with part-of-speech information added), morpheme analysis result (original text divided into morpheme units and with part-of-speech information added) mor ′ _t
Output: Weight of selected pseudo text tex ' _{t, y} W _{t, y}
Processing content: The selected pseudo-text tex ′ _{t, y} having the same part-of-speech word order as the part-of-speech word order of the T original texts tex _t is calculated with a larger weight W _{t, y} (s370). 'Output to the language model generation unit 140 together with _{t and y} . As a calculation method of the weight W _{t, y} , for example, there are the following methods.

The weights W _{t, y} are calculated using the appearance part-of-speech string set and the part-of-speech n-gram probability used in the second embodiment. However, the part-of-speech n-gram probability takes a value from 0 to 1. If the part-of-speech n-gram probability is large, it means “probably in word order”, and if the part-of-speech n-gram probability is small, it means “probably in word order”.

The pseudo text weight calculation unit 370 extracts the word order of the part of speech from the selected pseudo text tex ′ _{t, y} to which the part of speech information is added. Hereinafter, three methods for determining the weight W _{t, y} will be described.

(1) First Determination Method The pseudo text weight calculation unit 370 determines the word order of any part of speech included in the appearance part-of-speech sequence set in the appearance part-of-speech sequence set storage unit 253, the word order of the part-of-speech of the pseudo text tex ′ _{t, u} , Is determined to be equal to or greater than a predetermined ratio (for example, 95%), and if they match, it is determined that the word order of the part of speech of the pseudo-text tex ′ _{t, u} is probable, and the weight W _{t, y} the value a large value a _1. If they do not match, the pseudo-text tex _'t, determines that _y is inherently incorrect sentence patterns, weights W _t, the value of _y smaller value A _2.

In the case of the following second determination method and third determination method, the pseudo text weight calculation unit 370 further includes a part of speech n-gram corresponding to the part of speech n-gram pattern obtained from the word order of the part of speech of the pseudo text tex ′ _{t, y.} The probability is extracted from the part-of-speech n-gram probability storage unit 259 in the pseudo text selection unit 250.

(2) Second determination method The extracted part-of-speech n-gram probability is compared with a predetermined threshold value X, and if it is greater than or equal to the threshold value X, it is determined that the word order of the part-of-speech is likely, and the value of the weight W _{t, y} a to a large value _{a 1.} If it is less than the threshold X is, the pseudo-text tex _'t, determines that _y is inherently incorrect sentence patterns, weights W _t, the value of _y smaller value A _{2. However,} it is _A 1> A _2. X, A ₁ and A ₂ are determined in advance so that the recognition accuracy of the development set is maximized. For example, X, A ₁ , and A ₂ are prepared experimentally so that various combinations of values are prepared and the recognition accuracy as a language model is improved. Note that, when X approaches 0, the word order of all parts of speech is allowed, so the meaning of weighting according to the word order of parts of speech is lost. In this example, the pattern is classified into two patterns _equal to or greater than the threshold value X or less than the threshold value X. However, N threshold _{values Xn} are provided (where N is an integer equal to or greater than 2, and n = 1, 2, .., N, and X ₁ <X ₂ <... <X _N ), (N + 1) patterns can be classified. By increasing the number of threshold _values , it is considered that the ability to express the weights W _{t, y} is improved and the performance of the language model is improved. On the other hand, the number of parameters determined in advance (threshold values X ₁ , X ₂ ,..., _XN and (N + 1) values A ₁ , A ₂ ,..., A _{N + 1} corresponding to (N + 1) patterns increases). Calculation cost increases.

  As a method for comparing with the threshold value, a method similar to that for the pseudo text selecting unit 250 can be used. That is, the comparison is made as follows.

    (I) The average value of the extracted part-of-speech n-gram probabilities is obtained, and the average value is compared with the threshold value X. When the average value is equal to or greater than the threshold value X, it is determined that the word order of the part of speech is likely.

    (Ii) Each of the extracted part-of-speech n-gram probabilities is compared with a threshold X, and if the M part-of-speech n-gram probabilities are equal to or greater than the threshold X, it is determined that the word order of the part of speech is likely.

(3) Third determination method In the first place, if the part-of-speech n-gram probability is large, it means “probably in word order”, and if the part-of-speech n-gram probability is small, it means “probably in word order”. An average value of the extracted part-of-speech n-gram probabilities is obtained, and the average value (or a value obtained by multiplying the average value by a predetermined value) is used as a weight.

<Language model generation unit 140>
The language model generation unit 140 receives the original text tex _t , the selected pseudo-text tex ′ _{t, y} and the weight W _{t, y} , calculates the n-gram probability according to the equation (1) or (2), and the language model Is generated (s140). The method for generating the language model is the same as in the second embodiment. However, in the formula (1) or (2) or the like, for each selected pseudo-text tex ′ _{t, y} , the weight W _{t, y} is used instead of the weight W.

<Effect>
With such a configuration, the same effect as that of the second embodiment can be obtained. Furthermore, it is possible to improve the accuracy of the language model by giving a large weight W _{t, y} to the selected pseudo-text tex ′ _{t, y} having a more certain word order.

<Modification>
Although the pseudo text weight calculation unit 370 is added to the language model generation device 200 of the second embodiment, it may be added to the language model generation device 100 of the first embodiment. In this case, the pseudo text weight calculation unit 370 and the language model generation unit 140 use the pseudo text tex ′ _{t, u} with part-of-speech information added instead of the selected pseudo text tex ′ _{t, y} . Therefore, instead of the morphological analysis unit 110 of the first embodiment, using the morphological analysis unit 210 of the second embodiment divides the original text tex _t into morphemes by assigning parts of speech to each morpheme divided morpheme and it outputs the analysis result mor _'t. In this case, the pseudo-text weight calculation unit 370 obtains a part-of-speech n-gram probability and stores it in a storage unit (not shown).

<Other variations>
Further, the present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
The language model generation apparatus described above can also be functioned by a computer. In this case, each process of a program for causing a computer to function as a target device (a device having the functional configuration shown in the drawings in various embodiments) or a process procedure (shown in each embodiment) is processed by the computer. A program to be executed by the computer may be downloaded from a recording medium such as a CD-ROM, a magnetic disk, or a semiconductor storage device or via a communication line into the computer, and the program may be executed.

100, 200, 300 Language model generation device 110, 210 Morphological analysis unit 120 Syntax analysis unit 130 Pseudo text generation unit 140 Language model generation unit 250 Pseudo text selection unit 251 First part of speech information acquisition unit 253 Appearance part of speech sequence set storage unit 255 Two-part-of-speech information acquisition unit 257 determination unit 258 probability calculation unit 259 probability storage unit 370 pseudo text weight calculation unit

Claims (6)

A pseudo-text generation unit that generates pseudo-text by rearranging a plurality of clauses having the same dependency destination using original text that is divided into morpheme units and added with dependency relationships of clauses;
Seek n-gram probabilities using the occurrence frequency of n-gram patterns in frequency and the pseudo text n-gram patterns in the original text, viewed contains a language model generator for generating a language model,
Part of speech information is added to each original morpheme in the original text.
A pseudo text selection unit that compares the word order of the part of speech of the original text with the word order of the part of speech of the pseudo text, and selects a pseudo text having a probable part of speech part;
The language model generation unit obtains an n-gram probability using the appearance frequency of the n-gram pattern in the original text and the appearance frequency of the n-gram pattern selected in the pseudo-text selection unit. Generate a model,
Language model generator. The language model generation device according to claim 1 ,
The pseudo-text selection unit
A first part-of-speech information acquisition unit that extracts part-of-speech information added to the original text;
An appearance part-of-speech sequence storage unit that stores an appearance part-of-speech sequence set that is a set in the word order of the part of speech of the original text;
A second part-of-speech information acquisition unit that extracts a word order of the part-of-speech of the pseudo-text from the part-of-speech information added to the pseudo-text;
A determination unit that selects the pseudo-text when the word order of the part-of-speech of the pseudo-text and the word order of any part-of-speech included in the appearance part-of-speech sequence set match a predetermined ratio or more,
Language model generator. The language model generation device according to claim 1 ,
The pseudo-text selection unit
A first part-of-speech information acquisition unit that extracts part-of-speech information added to the original text;
An appearance part-of-speech sequence storage unit that stores an appearance part-of-speech sequence set that is a set in the word order of the part of speech of the original text;
A second part-of-speech information acquisition unit that extracts a word order of the part-of-speech of the pseudo-text from the part-of-speech information added to the pseudo-text;
A part-of-speech n-gram probability calculator for calculating a part-of-speech n-gram probability for a part-of-speech n-gram pattern included in the appearance part-of-speech sequence set;
A part-of-speech n-gram probability storage unit for storing the part-of-speech n-gram probability;
The part-of-speech n-gram probability corresponding to the part-of-speech n-gram pattern obtained from the word order of the part-of-speech part of the pseudo-text is extracted from the part-of-speech n-gram probability storage unit, and the extracted part-of-speech n-gram probability and a predetermined threshold value are obtained. A determination unit that selects pseudo-text corresponding to the word order of the part of speech if the comparison is greater than or equal to the threshold,
Language model generator. The language model generation device according to any one of claims 1 to 3 ,
Part of speech information is added to each original morpheme in the original text.
A language model weight calculation unit that calculates a greater weight for pseudo text having more part-of-speech word order than part-of-speech word order of the original text,
The language model generation unit weights the appearance frequency of the n-gram pattern in the original text and the appearance frequency of the n-gram pattern in the pseudo text with the weight, obtains an n-gram probability, Generate,
Language model generator. A pseudo-text generation step of rearranging a plurality of clauses having the same dependency destination to generate pseudo-text by using original text that is divided into morpheme units and to which a dependency relationship of clauses is added;
Using said frequency of n-gram patterns sought n-gram probability in occurrence frequency and the pseudo text n-gram patterns in the original text, viewed contains a language model generating step of generating a language model,
Part of speech information is added to each original morpheme in the original text.
A pseudo-text selecting step of comparing the word order of the part of speech of the original text with the word order of the part of speech of the pseudo-text to select a pseudo-text having a probable part-of-speech word order;
In the language model generation step, an n-gram probability is obtained using the appearance frequency of the n-gram pattern in the original text and the appearance frequency of the n-gram pattern in the pseudo text selected in the pseudo text selection step. Generate a model,
Language model generation method. The program for functioning a computer as a language model production | generation apparatus in any one of Claims 1-4 .

Images