JP5180522B2 - Machine translation apparatus, machine translation method, program thereof, and recording medium - Google Patents

Description

The present invention relates to a machine translation device, a machine translation method, a program thereof, and a recording medium.

Conventionally, a technique (statistical machine translation) that realizes machine translation using a statistical model is known (for example, Non-Patent Document 1, Non-Patent Document 2, and Non-Patent Document 3).
Statistical machine translation is formulated as a problem of searching for a target language word string (sentence) that maximizes the translation probability when a source language word string (sentence) is given. Here, when the translation probability is expressed by a logarithmic linear model, statistical machine translation is finally formulated by Equation (1).

  Expression (1) can also be rewritten as Expression (2) using a vector.

In the word string of the translation source language, identification codes of integers “1” to “J” are assigned to the respective word positions. That is, the word string in the source language is composed of “J” words. When the translation source language is Japanese, the translation source language word string is composed of “J” independent words, attached words, and punctuation marks (.,). Also, in the word string of the translation destination language, identification codes of integers “1” to “I” are assigned to the respective word positions. That is, the word string in the translation target language is composed of “I” words. M (1 ≦ m ≦ M) represents an integer for identifying a feature function, and M represents the number of feature functions. Each feature function represents a certainty as a translation, a certain one as a translation target language, or the like. The word string set E represents all word string sets that can be generated by any combination of words in the translation target language. Each feature weight λ _m is set such that the translation accuracy value in the feature weight learning parallel translation corpus is maximized by using an error minimization learning method or the like (for example, see Non-Patent Document 4).

For translation from a source language sentence to a destination language sentence, a weighted synchronous context-free grammar is used to determine whether the source language sentence and the destination language sentence are A method for modeling the association is known (for example, see Non-Patent Document 2 and Non-Patent Document 3).
The weighted synchronous context free grammar is composed of a set of weighted generation rules shown in Expression (3).

  Here, X on the left side of the arrow indicates a non-terminal symbol. Γ on the right side of the arrow is a symbol string composed of terminal symbols or non-terminal symbols and corresponds to the source language. Α is a symbol string composed of a terminal symbol or a non-terminal symbol, and corresponds to the language to be translated. “˜” represents a one-to-one correspondence between a non-terminal symbol included in the symbol string γ and a non-terminal symbol included in the symbol string α. Here, it is assumed that the number of non-terminal symbols included in the symbol string γ is the same as the number of non-terminal symbols included in the symbol string α.

Table 1 shows a specific example of the generation rule shown in Expression (3). Here, X _(k) is a non-terminal symbol, and k represents a correspondence relationship of the non-terminal symbol.

  In the conventional modeling using the weighted synchronous context free grammar, the derivation D of the weighted synchronous context free grammar is used to convert the word string in the source language and the word string in the target language into f (D) and e, respectively. (D) is described. Here, for example, if the generation rule covering the “j” -th word from the “i” -th word in the word string f (D) of the translation source language is r, the derivation D is a triple <r, i, j>. It is represented by a set of

  In the modeling of Non-Patent Document 2, Formula (4) obtained by modifying Formula (1) formulated from statistical machine translation to a derivation base is used. In this formulation, when a translation source language word string is given, a derivation D ^ that maximizes the linear sum of the product of the feature function and the feature weight is obtained, and the corresponding e (D ^ ) Is the translation result. Here, the symbol “＾ (hat)” indicates a symbol added on the letter “D”, and the symbol “＾ (hat)” is used in the same meaning hereinafter. Note that s. t. Stands for such that.

Various variations can be considered as to what is used for the value h _m (D) of each feature function shown in Expression (4). For example, the natural logarithm log _e of the following six function values may be used (see Non-Patent Document 1, for example). The values of these six functions are the translation probabilities P _{e | f} (D) and P _{f | e} (D) shown in the equations (5) and (6), and the equations (7) and (8). The lexical weights Lex _{e | f} (D), Lex _{f | e} (D), the probability P _LM (e (D)) of the n-gram language model, and the phrase penalty exp (length (e (D))). . Here, length (·) indicates a function that returns the number of words.

The translation probabilities P _{e | f} (D), P _{f | e} (D) and the lexical weights Lex _{e | f} (D), Lex _{f | e} (D) are values for evaluating the likelihood of translation. It is also called a translation model. Specifically, for example, the translation probability P _{e | f} (D) is obtained by using the probability P (α | γ) for each production rule r included in the derivation D as the score for each production rule, as shown in Expression (5). , The scores for each production rule are integrated for all production rules r included in derivation D.

For example, in the method described in Non-Patent Document 2, a solution search in translation is performed according to the following procedure. First, in the bottom-up syntax analysis based on the CKY (Cocke-Kasami-Younger) method, the generation rules on the source language side of the synchronous context free grammar are applied to the source language word string, and the source language syntax analysis tree Get. Then, an optimum derivation D ^ of the synchronous context free grammar corresponding to the parse tree of the source language is obtained based on the above-described equation (4), and the word string of the translation destination language is obtained based on the optimum derivation D ^. Is generated. However, since there are a large number of solution candidates (hereinafter referred to as hypotheses) in the statistical machine translation solution search, it is virtually impossible to perform a full search to find the true optimal solution from the viewpoint of computational complexity. It has become. Therefore, conventionally, a sub-optimal solution is obtained by performing processing while executing predetermined pruning on the derivation D of the subtree of the synchronous context free grammar partially configured in the solution search process.
Philipp Koehn, Franz Josef Och, and Daniel Marcu, Statistical phrase-based translation, In Proc. Of NAACL 2003, p. 48-54, Edmonton, Canada, 2003 David Chiang, A hierarchical phrase-based model for statistical machine translation, In Proc. Of ACL 2005, p. 263-270, Ann Arbor, Michigan, June 2005 Taro Watanabe, Hajime Tsukada, and Hideki Isozaki, Left-to-right target generation for hierarchical phrase-based trans1ation, In Proc. Of COLING / ACL2006, p. 777-784, Sydney, Australia, Jully 2006 Franz Josef Och, Minimum error rate training in statistical machine translation, In Proc. Of ACL 2003, p. 160-167, Sapporo, Japan, July 2003

In the prior art described above, the correspondence between the subtree in the source language and the subtree in the target language is such that the translation probabilities P _{e | f} (r), P _{f |} defined for each rule r of the synchronous context-dependent grammar. Modeled by _e (r) and lexical weights Lex _{e | f} (r), Lex _{f | e} (r). However, neither the translation source nor the translation destination is explicitly modeled as to which rule is likely to expand the non-terminal symbols, and is indirectly restricted by the language model of the translation target language. It ’s just that. In particular, the methods described in Non-Patent Document 2 and Non-Patent Document 3 are based on a synchronous context-dependent grammar that is automatically acquired, and except for the glue rule, there is actually only one type of non-terminal symbol. The roughness of modeling is serious.

12 and 13 show examples of mistranslation and correct translation of a conventional method using a synchronous context-dependent grammar as specific examples of how it is likely to expand the non-terminal symbols described above. . This example shows the following sentence for translation from Japanese to English:
(1) Sentence in the source language (Japanese):
"Japan's protest against China's measures is natural."
(2) Destination language (English) sentence (example of mistranslation):
"It is natural for China's action to protest Japan."
(3) Translation language (English) sentence (correct translation example):
"It is natural for Japan to protest China's action."

As shown in mistranslation example of FIG. 12, the translation of the original language side of the "X → X ₍₁₎ of course it is X is _(2)" X ₍₁₎ is, "X ₍₂₎ with respect to X → X _(1)" The rules are expanded.
On the other hand, as shown in the correct translation example of FIG. 13, the translation of the original language side of the "X → X ₍₁₎ is a naturally X _(2)" X ₍₁₎ is, of Japan for "X → X ₍₁₎ X ₍₂₎ ”rule.
In order to make the score of the correct translation example (3) more advantageous (higher) than the score of the incorrect translation example (2), it is modeled that the development by the latter rule is more likely. I think it should be done. However, each conventional method does not explicitly perform such modeling. Therefore, the conventional method has a problem that the translation accuracy is low. In FIG. 12 and FIG. 13, the non-terminal symbols are represented by “X1, X2, X8, X6,.

  In general, words that do not correspond to the translation source word may appear in the translation result. For example, in Japanese, the subject is often omitted. For this reason, in Japanese-English translation, there may be a case where there is no word in the source language “Japanese” corresponding to the subject of the target language “English”. Similarly, since there is no article in Japanese, there is often no word in the source language “Japanese” corresponding to the article in the target language “English”. In such a case, in order to make the score of the correct translation example more advantageous (higher) than the score of the incorrect translation example, a word that appears in the translation result and has no word corresponding to the translation source word is modeled as a feature. I think it should be done. However, each conventional method does not explicitly perform such modeling. Therefore, the conventional method has a problem that the translation accuracy is low.

  Therefore, an object of the present invention is to provide a machine translation technique that can solve the above-described problems and improve translation accuracy.

In order to solve the above-described problem, the machine translation device according to the present invention provides a word defined from the upper and lower relations on the syntax tree constituting the word string of the translation source language or the word string of the translation destination language in the parallel translation learning data. A hierarchical feature that defines a binary feature for each pair , and the translation when the word corresponding to the word constituting the word sequence of the translation target language in the parallel translation learning data is not included in the word sequence of the source language A pair of a word inserted in the word string of the destination language and each word included in the word string of the translation source language is represented by one of the binary values, and the other of the binary values for the other word pairs Translations entered using at least one of the translation target language insertion features expressed in (1) above and a translation model that defines the likelihood of correspondence between the source language word string and the target language word string The input which is the translation result of the original language word string A hypothesis that is a partial hypothesis that is finally generated by sequentially creating a new partial hypothesis longer than the predetermined partial hypothesis and expanding the predetermined partial hypothesis as a word string in the target language corresponding to A weight corresponding to a feature including at least one of the hierarchical feature and the translation target language insertion feature and the translation model, based on parallel feature learning data for feature weight learning A feature weight learning means for storing learning results as feature weights in a storage means, a feature vector including at least one of the hierarchical feature and the translation target language insertion feature and the translation model as elements. Partial hypothesis score calculating means for calculating an inner product with a weight vector indicating the feature weight as a partial hypothesis score indicating an evaluation value of the created partial hypothesis; The partial hypothesis score among the partial hypotheses finally generated by searching for a predetermined partial hypothesis applicable to the input source language word string and expanding the predetermined partial hypothesis And a hypothesis search means for searching for a partial hypothesis with the maximum as the hypothesis.

In order to solve the above problem, the machine translation method according to the present invention is defined from upper and lower relations on a syntax tree that constitutes a word string of a translation source language or a word string of a translation destination language in parallel translation learning data. A hierarchical feature in which a binary feature is defined for each word pair to be translated, and a word corresponding to a word constituting a word string of a translation target language in the parallel translation learning data is not included in the word string of the translation source language Represents a pair of a word inserted into the word string of the translation-destination language and each word included in the word string of the translation-source language , one of the binary values, and a binary value for the other word pairs Input using at least one of the translation target language insertion features represented by the other one and a translation model that defines the likelihood of correspondence between the source language word string and the target language word string Is the translation result of the translated source word string A partial hypothesis that is finally generated by sequentially creating a new partial hypothesis from a predetermined partial hypothesis and extending the predetermined partial hypothesis as a word string of the translation target language corresponding to the input A machine translation method of a machine translation apparatus that outputs a hypothesis, wherein a feature weight learning unit corresponds to a feature including at least one of the hierarchical feature and the translation target language insertion feature and the translation model. A feature weight learning step that learns weights based on parallel feature learning data for feature weight learning and stores a learning result as a feature weight in a storage unit; and a partial hypothesis score calculation unit that inserts the hierarchical features and the translation target language An inner product of a feature vector including at least one of the features and the translation model as elements and a weight vector indicating the feature weight is calculated as the product. A partial hypothesis score calculating step for calculating a partial hypothesis score indicating an evaluation value of the received partial hypothesis, and a hypothesis search means for searching for a predetermined partial hypothesis applicable to the input translation source language word string. A hypothesis search step of searching for a partial hypothesis having the maximum partial hypothesis score among the partial hypotheses finally generated by extending the predetermined partial hypothesis as the hypothesis. To do.

  According to the machine translation device configured as described above or the machine translation method according to such a procedure, the machine translation device assigns a weight corresponding to a feature including at least one of a hierarchical feature and a translation target language insertion feature and a translation model, Learning is performed based on parallel weight learning data for feature weight learning. Here, the features may further include a language model and a phrase penalty. Hierarchical features are features that represent the hierarchical features of the subtrees that make up the translation source or destination sentence of the correct answer data, so by learning the feature weight in addition to the features previously used Thus, it becomes possible to generate a translation sentence in a manner that is likely to be developed as a correct translation for the input translation source sentence. In addition, because the target language insertion feature is a feature that expresses a word inserted in the target sentence that is not included in the source sentence of the correct answer data, this feature was added to the previous feature. By learning the weight of the feature, for example, when translating from a language that often omits the subject, such as Japanese, or a language that does not have an article, to a language that has an article, it is likely to develop as a correct translation It becomes possible to generate a translation sentence in a manner.

  The machine translation device uses higher-dimensional features and weights to perform more precise modeling compared to the conventional case where scores are calculated using only translation models, language models, and phrase penalties as features. can do. The machine translation apparatus can improve the translation accuracy of the word string in the translation target language as compared to the case of learning weights using only the translation model, language model, and phrase penalty as features as in the conventional apparatus.

  A machine translation program according to the present invention causes a computer to execute the above-described machine translation method. By being configured in this way, a computer in which this program is installed can realize each function based on this program.

A computer-readable recording medium according to the present invention is characterized in that the machine translation program described above is recorded. By being configured in this way, a computer equipped with this recording medium can realize each function based on a program recorded on this recording medium.

  According to the present invention, translation accuracy can be improved.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the best mode for carrying out a machine translation device and a machine translation method of the present invention with reference to the drawings, and a production rule creation device and a production rule creation method according to a reference example of the present invention (hereinafter referred to as “embodiment”) Will be described in detail. Hereinafter, the production rule creation device and the production rule creation method, and the machine translation device and the machine translation method will be described in order.

[Configuration of generation rule creation device]
FIG. 1 is a block diagram showing a configuration of a production rule creation device according to a reference example of the present invention.
The generation rule creation device 1 creates features and generation rules used in a machine translation device that mechanically translates a word string in a translation source language into a word string in a translation destination language. In the following description, it is assumed that the source language is Japanese and the target language is English.
The generation rule creation device 1 is composed of, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an input / output interface, and the like. As shown in FIG. 1, input / output means 10, storage means 11, and control means 12 are provided.

The input / output means 10 inputs the bilingual corpus (parallel translation learning data) 150 to the control means 12, and outputs the result of the arithmetic processing, data stored in the storage means 11, and the like to the output device D. Further, the input / output means 10 inputs a predetermined command (mode selection command, operation command, etc.) from the input device K to the control means 12. In this embodiment, the mode selection command includes a command for selecting the “word correspondence creation” mode, a command for selecting the “rule table creation” mode, and a command for selecting the “language model creation” mode.
The bilingual corpus 150 includes a plurality of data of combinations of translation source language word strings and translation destination language word strings having the same meaning.

  The storage unit 11 includes, for example, a RAM used for arithmetic processing by the CPU, and a ROM and HDD for storing predetermined programs, various databases, processing results, and the like. For example, in the storage unit 11, as a processing result, a word correspondence 111, a phrase pair 112, a rule 113, a rule table 114, a language model 115, a translated language bigram feature 116, a word pair feature 117, A translation target language insertion feature 118 and a hierarchical feature 119 are stored.

  The control unit 12 includes a mode determination unit 121, a language model learning unit 122, a word correspondence creation unit 123, an inter-language correspondence feature extraction unit 124, a phrase pair extraction unit 125, a generation rule creation unit 126, a translation score Computation means 127 and hierarchical feature extraction means 128 are provided.

  The mode determination unit 121 determines the mode indicated by the mode selection command input from the input device K via the input / output unit 10. When the mode is the “word correspondence creation” mode, the mode determination unit 121 instructs the word correspondence creation unit 123 to input the bilingual corpus 150. Further, when the mode is the “rule table creation” mode, the mode determination unit 121 instructs the phrase pair extraction unit 125 to input the bilingual corpus 150. In addition, when the mode is the “language model creation” mode, the mode determination unit 121 instructs the language model learning unit 122 to input the translation destination language corpus 140.

  The language model learning unit 122 reads the translation destination language corpus 140, calculates an n-gram representing the certainty as the translation destination language, and stores the n-gram in the language model 115. Further, the language model learning unit 122 extracts the translation destination language bigram feature 116 from the read translation destination language corpus 140. Details of the translated language bigram feature 116 will be described later.

  The word correspondence creation unit 123 uses a statistic about word co-occurrence obtained from the bilingual corpus 150 when a command for selecting the “word correspondence creation” mode is input, and provides many-to-many (translation source language or translation destination). The word correspondence 111 (including that the language word does not correspond anywhere) is automatically obtained. In order to obtain the many-to-many word correspondence 111, for example, the word correspondence creating unit 123 obtains the optimum one-to-many correspondence and many-to-one correspondence for the entire sentence by using a translation model in units of words. Combine both. As an example of the combination method, there is a method of using an intersection of a one-to-many correspondence and a many-to-one correspondence, and adding a neighboring element of a one-to-many correspondence and a many-to-one correspondence (non-patent document). 1). For the translation model for each word, see `` Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer, The mathematics of statistical machine translation: Parameter estimation, Computatinal Linguistics, 19 (2 ): 263-311, 1993 ". In addition, when the bilingual corpus 150 itself holds information about the word correspondence 111, the word correspondence creation unit 123 may be omitted.

FIG. 2 shows an example of word correspondence of Japanese-English parallel translation. Eight words and black circles indicating punctuation points (periods) correspond to words.
“The Federal Constitutional Court decides on the issue of unconstitution.”
"The Federal Constitutional Court decides on the question of unconstitutionality."

The inter-language correspondence feature extraction unit 124 includes a word pair feature extraction unit 131 and a translation target language insertion feature extraction unit 132.
The word pair feature extraction means 131 extracts the word pair feature 117 from the word correspondence 111. The word pair feature 117 is a feature based on the word pair of the translation source language / translation destination language.
The translation target language insertion feature extraction unit 132 extracts the translation target language insertion feature 118 based on the word correspondence 111.
The translation destination language insertion feature 118 is inserted into the word sequence of the translation target language when the word corresponding to the word constituting the translation target language word sequence is not included in the translation source language word sequence. The feature which expresses the relationship between the word and the word contained in the word sequence of the translation source language is shown. Details of the word pair feature 117 and the translated language insertion feature 118 will be described later.

  The phrase pair extraction unit 125 obtains a combination of words or phrases having the same meaning in the source language and the target language from the parallel corpus 150 based on the word correspondence 111 between the source language word and the target language word. The phrase pair 112 is extracted. The extracted phrase pair 112 is stored in the storage unit 11. When a word correspondence a is calculated for the parallel translation sentence of the translation source language word string and the translation destination language word string shown in Expression (1), the phrase pair extraction unit 125 calculates Expression (9). Extract the indicated phrase pair. Here, the word correspondence a is a set of a set of word positions in the translation destination language and word positions in the translation source language. “I, m, j, n” in Expression (9) represents an integer, and satisfies the relationship of Expression (10) to Expression (12) with the word correspondence a.

For example, a phrase pair as shown in Table 2 is extracted by the phrase pair extraction unit 125 from the word correspondence of the parallel translation shown in FIG.
At this time, the correspondence of each word in the phrase pair, for example, the word correspondence such as (unconstitutionality) and (question, question) in the first (first line) phrase pair shown in Table 2, is also the phrase pair 112. Are stored at the same time. That is, Table 2 is actually a column in which word correspondences such as (unconstitutionality), (question, question) are stored in each row in addition to the column in which the phrase pair is stored (illustration is omitted). Is provided.

The generation rule creation means 126 is based on a list of phrase pairs stored in the phrase pair 112 extracted from the parallel translation pair for each parallel translation pair (combination of parallel translations) in the parallel corpus 150. A rule for generating a synchronous context free grammar is created and stored in the rule 113. Here, in the generation rule to be created, there is a restriction that the translation destination language side must start with a terminal symbol. In addition, the generation rules stored in the rule 113 allow duplication. In the generation rule created in this way, the following glue rule used in Non-Patent Document 2 is not used. The glue rule indicates the following two rules when S is a start symbol and X is a non-terminal symbol.
S → <S ₍₁₎ X ₍₁₎ , S ₍₁₎ X ₍₁₎ >
S → <X ₍₁₎ , X ₍₁₎ >

に対応して、式（１３）〜式（１７）の生成規則を生成する。このうち、式（１４）〜式（１７）の生成規則は、式（１３）の生成規則から自動的に生成することができる。また、式（１４）〜式（１７）の生成規則に付与されるスコアについても、式（１３）の生成規則と同一値を用いることができる。このような理由から、実装上は、式（１４）〜式（１７）の生成規則は明示的にストレージに格納する必要がない。式（１４）〜式（１７）の生成規則は、非特許文献２で用いられるグルー規則の非終端記号Ｘを、Ｘを左辺とする個々の規則で１回書き換えたものに対応している。 The generation rule creation means 126 is a phrase pair

Corresponding to the above, generation rules of Expression (13) to Expression (17) are generated. Among these, the production | generation rule of Formula (14)-Formula (17) can be automatically produced | generated from the production | generation rule of Formula (13). Moreover, the same value as the production | generation rule of Formula (13) can be used also about the score provided to the production | generation rule of Formula (14)-Formula (17). For this reason, it is not necessary to explicitly store the generation rules of Expressions (14) to (17) in the storage for implementation. The production | generation rule of Formula (14)-Formula (17) respond | corresponds to what rewritten the non-terminal symbol X of the glue rule used by the nonpatent literature 2 once with each rule which uses X as a left side.

  When the generation rule shown in Expression (18) is generated and the phrase pair on the right side of Expression (13) satisfies the relationship of Expression (19) and Expression (20), the generation rule creating unit 126 satisfies Expression (21). ) Is generated.

  Here, α has a restriction that it must start with a terminal symbol. Although not essential to the present embodiment, the generation rule creation unit 126 also adopts the following restrictions, for example. First, both γ and α must contain at least one terminal symbol. Second, production rules can have a maximum of two nonterminal symbols. However, nonterminal symbols must not be adjacent in the source language.

The translation score calculation means 127 counts the generation rules stored in the rule 113 with duplication allowed, the translation probabilities P _{e | f} (r) and P _{f | e} (r) corresponding to the generation rules r, and the lexical The weights Lex _{e | f} (r) and Lex _{f | e} (r) are calculated, and the calculation results are stored as features in the rule table 114 in association with the generation rules r. FIG. 3 shows an example of the rule table. Table 3 shows calculation formulas for each translation probability and lexical weight corresponding to the generation rule r. This score calculation is based on Non-Patent Document 2.

  As with the phrase pair 112, for the rules stored in the rule 113 and the rule table 114, the correspondence between the translation source word and the translation destination word is also stored together. Even in the same rule, a plurality of variations of word correspondence can be considered, but for each rule, the word correspondence that appears most frequently in the rule 113 is used in the rule table 114.

  The hierarchical feature extraction unit 128 indicates a feature that expresses a hierarchical feature of a sub-tree constituting a word string of the translation source language or a word string of the translation destination language when the generation rule creation unit 126 creates the rule table 114. The hierarchical feature 119 is extracted as one of the features. In the present embodiment, a feature that expresses a hierarchical feature of a subtree constituting a word string in the translation source language is extracted as a hierarchical feature 119. Details of the hierarchical feature 119 will be described later. Note that the hierarchical feature extraction unit 128 can be configured to extract the hierarchical feature 119 based on the word correspondence in the rule table 114 without depending on the operation of the generation rule creation unit 126.

[Concrete example]
Specifically, the generation rule creation unit 126 generates a rule (only the right side is shown) as shown in Table 4 from the parallel translation shown in FIG. In the example of Table 4, the translation destination language side on the right side of each production rule always starts with a terminal symbol (word). In addition, Table 4 actually includes a column (not shown) storing word correspondence such as (Federal) in addition to a column storing the right side of each generation rule in each row.

  As features assumed in the above formula (1) and the above formula (2), those having real values such as translation probabilities and lexical weights shown in Table 3 have been conventionally used. In the present embodiment, not only such a feature that takes a real value but also a binary feature that takes a simple value of “0” or “1” as shown in Expression (22), for example, is used.

ここで、ｉは多次元の素性ベクトルｈ（ｆ，ｅ）の次元をあらわす。

Here, i represents the dimension of the multidimensional feature vector h (f, e).

  It is not self-evident that what kind of binary feature is added to the feature that has been used conventionally will ultimately improve the translation accuracy. Therefore, in the present embodiment, as shown below, it is configured so that four types of binary features can be used.

<Word pair features>
The word pair feature 117 (see FIG. 1) will be described with reference to FIG. Here, as shown in FIG. 4, a phrase pair 403 (403a, 403b, 403c) is extracted from the word string 401 of the translation destination language and the word string 402 of the translation source language, and stored in the phrase pair 112. It shall be. For example, it is assumed that word correspondences 404, 405, and 406 included in the phrase pair 403b are stored in the word correspondence 111. Here, the word correspondence 404 is (e _i , f _{j + 1} ), the word correspondence 405 is (e _i , f _{j + 1} ), and the word correspondence 406 is (e _{i + 3} , f _j ). At this time, for each word correspondence (w _e , w _f ), a feature h _i (f, e) shown in Expression (23) is defined. The word correspondence 407 included in the phrase pair 403a is (e _i-1 , e _j-1 ).

The feature shown in the equation (23) can be regarded as a kind of translation model.
In order to realize this feature, the word pair feature extraction unit 131 (see FIG. 1) of the inter-language correspondence feature extraction unit 124 adds (e _i , f _{j + 1} ), (e _{i + 2} ) to the word pair feature 117. , F _{j + 2} ), (e _{i + 3} , f _j ).

Further, the word pair feature 117 is not limited to the feature represented by the above-described formula (23), and further, the one based on the feature based on the word pair bigram that expresses the concatenation relationship of the word pairs may be used. Good. Here, the feature is defined by expressing the connection relation of the word pairs in the order of the translation destination language. For example, as shown in FIG. 4, based on the order of the word string 401 in the translation destination language, the order as indicated by the arrows is defined for each word in the word string 402 in the translation source language. In this example, ((e _i−1 , f _j−1 ), (e _i , f _{j + 1} ), (e _i , f _{j + 1} ), (e _{i + 2} , f _{j + 2} ), ( A word pair bigram like e _{i + 2} , f _{j + 2} ), (e _{i + 3} , f _j )) can be defined. Therefore, for the word pair bigram ((e ₁ , f ₁ ), (e ₂ , f ₂ )), a feature h _i (f, e) shown in Expression (24) is defined.

When using the word pair bigram feature using the feature shown in the above equation (24), the word pair feature extracting unit 131 (see FIG. 1) converts ((e _i−1 , f _j− ) into the word pair feature 117. ₁ ), (e _i , f _{j + 1} ), (e _i , f _{j + 1} ), (e _{i + 2} , f _{j + 2} ), (e _{i + 2} , f _{j + 2} ), (e _{i +3} , f _j )) is added. In this way, by defining the secondary features in the order of the translation target language in addition to the word correspondence, the word rearrangement can be modeled simultaneously with the certainty as the translation of the word. Further, as shown in FIG. 4, for example, a word pair bigram ((e _i−1 , f _j−1 ), (e _i , f _{j + 1} )) straddling the boundary line between the phrase pair 403a and the phrase pair 403b. Can also be used to model phrase reordering.

<Destination language insertion feature>
The translation target language insertion feature 118 (see FIG. 1) models a model that is present in the translation target language but has no word corresponding to the translation source language. In general, in the translation result, there may be a case that does not correspond to the translation source word. For example, in Japanese, the subject is often omitted. For this reason, in Japanese-English translation, there may be a case where there is no word in the translation source language “Japanese” corresponding to the subject of the translation target language “English”. Similarly, since there is no article in Japanese, there is often no word in the source language “Japanese” corresponding to the article in the target language “English”. The translation target language insertion feature 118 (see FIG. 1) considering such background will be described with reference to FIG.

As shown in FIG. 4, for example, in the word string 401 of the translation target language, the word corresponding to the word e _{i + 1} does not exist in the word string 402 of the translation source language. In the present embodiment, the features associated with all the words in the translation source language are defined for such words in the translation destination language. For example, word pairs such as (e _{i + 1} , f ₁ ),..., (E _{i + 1} , f _J ) can be defined for the translation source language f ₁ ^J composed of J words. Then, for each word pair (w _e , w _f ), a feature h _i (f, e) shown in Expression (25) is defined.

In order to show a specific example of the translation target language insertion feature 118 using the feature h _i (f, e) defined by the equation (25), it is assumed that the correct translation data shown in FIG. Assume. In FIG. 13, solid arrows indicate word correspondences. For example, English “for” does not have a corresponding Japanese word, so from this example, the word pairs (for, China), (for, of), (for, measures), (for, for) , (For, Japan), (for, protest), (for, is), (for, naturally), (for, da) and (for,.) Are defined as 10 features.

  When this translation destination language insertion feature 118 is used, the translation destination language insertion feature extraction unit 132 (see FIG. 1) of the inter-language correspondence feature extraction unit 124 adds the translation target language insertion feature 118 to the word pair (for, China). , (For,), (for, measures), (for, against), (for, Japan), (for, protest), (for, is), (for, of course), (for, is) and ( for,.) is added. Conversely, a deletion feature indicating that there is no translation destination language corresponding to the translation source language side is also conceivable. In this case, the decoder needs to calculate a feature for a translation target word that has not yet been generated, and decoding is complicated, so that it is not used in this embodiment.

<Destination language bigram features>
The translated language bigram feature 116 (see FIG. 1) is a feature used to express the fluentness of the translated language, and reinforces the conventional language model feature. For example, as shown in FIG. 4, in the word string 401 of the translation destination language, the word pairs (e _i−1 , e _i ), (e _i , e _{i + 1} ), (e _{i + 1} , e _{i + 2} For each word pair (e ₁ , e ₂ ) such as),..., A feature h _i (f, e) shown in Expression (26) is defined.

When this translated language bigram feature 116 is used, the language model learning means 122 (see FIG. 1) adds the word pairs (e _i−1 , e _i ), (e _i , e _{i +} ) to the translated language bigram feature 116. ₁ ), (e _{i + 1} , e _{i + 2} ).

<Hierarchical features>
The hierarchical feature 119 (see FIG. 1) is a feature that defines upper and lower relations of the synchronous context free grammar. For example, in the rules r1 and r2 of the synchronous context free grammar created by the generation rule creation means 126 (see FIG. 1), when the nonterminal symbol of r1 is expanded at r2, the rule set (r1, r2) is The feature h _i (f, e) shown in equation (27) is defined.

  In addition, as a simpler feature definition, there is a variation that uses only the rules of the translation source or destination of the synchronous dependency grammar.

  When the hierarchical feature based on the synchronous context free grammar rule as shown in the equation (27) is used, the hierarchical feature extraction unit 128 (see FIG. 1) adds the rule (r1, r2) to the hierarchical feature 119. to add.

If features are defined by a combination of rules, the number of features may be too large. Therefore, in this embodiment, in order to reduce the number of features, the hierarchical features are defined by the upper and lower relations on the syntax tree for the words of the translation source or the translation destination. The hierarchical features in this case will be described with reference to FIG. FIG. 5 shows an example of a tree structure on the translation source language side. As shown in FIG. 5, the non-terminal symbol X ₍₁₎ denoted by reference numeral 501 is derived from the derivation rules “X ₍₁₎ → f _j−1 X ₍₂₎ f _{j + 3} ”, “X ₍₂₎ → f _j f _{j + 1} X ₍₃₎ ”,“ X ₍₃₎ → f _{j + 2} ”shows a structure expanded as shown in the order of reference numerals 502 to 504. In this example, word pairs (f _j−1 , f _j ), (f _j−1 , f _{j + 1} ) are obtained as word pairs obtained by combining all words of the parent rule from all words of the parent rule. , (F _{j + 3} , f _j ), (f _{j + 3} , f _{j + 1} ), (f _j , f _{j + 2} ), (f _{j + 1} , f _{j + 2} ) can be defined. For each such word pair (f ₁ , f ₂ ), a feature h _i (f, e) shown in Expression (28) is defined.

  Furthermore, as a device for reducing the number of features, a hierarchical feature can be defined by limiting to words in the source language associated with the target language. In order to show a specific example of the hierarchical feature defined in this way, it is assumed that the correct translation data shown in FIG. 13 is present in the learning data. In this case, the features are defined corresponding to the word pairs (for, China), (Japan, China), (for, protest), and (Japan, protest). In this embodiment, hierarchical features based on words (word-based hierarchical features) are defined in this way. Therefore, the hierarchical feature extraction unit 128 stores the word pairs (for, China), (Japan, China), (for, protest), and (Japan, protest) in the hierarchical feature 119. In this description, the hierarchical feature is defined on the translation source language side. Similarly, the hierarchical feature can also be defined on the translation target language side.

<Regarding word normalization>
Conventional features assumed the surface form of words. However, there is a possibility that an overlearning problem may be caused only by the feature assuming such a surface layer shape. Note that the phenomenon in which the generalization error for data not used for learning becomes large is called overlearning. In the present embodiment, normalized words are used in combination to avoid overlearning. Although the normalization method is not limited, for example, the following method can be used.

(1) Word class—Learning a class for each word by clustering and letting that class be the normal form of the word.
(2) Part of speech-Let the part of speech given by the morphological analysis system be the normal form.
(3) Prefix / Suffix-4-character prefix or suffix is in normal form. For example, in English, “violate” is normalized as “viol +” and “+ late” by taking a prefix / suffix of 4 characters.
(4) Normalize various surface types by stem-stemming algorithm.
(5) Numbers-normalize numbers. For example, “2007/6/27” is normalized as “@@@@ / @ / @@” by replacing the numeric part with “@”.

  In the present embodiment, possible normalization of all words appearing in all the features in the learning data is considered, and the features of the normalized words are added. For example, a feature such as (thre +, threat) (+ reat, threat) can be added to the word pair feature (threat, threat) by normalizing with a 4-character prefix / suffix. The normalization shown here is an example, and normalization suitable for each language to be translated can be easily incorporated.

  The mode determination unit 121, the language model learning unit 122, the word correspondence creation unit 123, the inter-language correspondence feature extraction unit 124, the phrase pair extraction unit 125, the generation rule creation unit 126, and the translation score calculation. The means 127 and the hierarchical feature extraction means 128 are realized by the CPU developing and executing a predetermined program stored in the HDD or the like of the storage means 11 on the RAM.

[Operation of generation rule creation device]
The operation of the production rule creation device shown in FIG. 1 will be described with reference to FIG. 6 (see FIG. 1 as appropriate). FIG. 6 is a flowchart showing the operation of the generation rule creation device shown in FIG.
The production rule creation device 1 determines the mode by the mode determination unit 121 (step S1). As a result of the determination, if the mode is the “word correspondence creation” mode, the generation rule creation device 1 inputs the bilingual corpus 150 to the word correspondence creation means 123 via the input / output means 10 (step S2). A word correspondence is created by the correspondence creating means 123 (step S3). The created word correspondence 111 is stored in the storage unit 11.

  Then, the generation rule creating device 1 extracts the word pair feature from the word correspondence 111 by the word pair feature extraction unit 131 (step S4), and inserts the translation destination language from the word correspondence 111 by the translation destination language insertion feature extraction unit 132. The feature 118 is extracted (step S5: translation destination language insertion feature extraction step). In addition, the process order of step S4 and step S5 is arbitrary, and may be processed in parallel.

  If the result of determination in step S1 is that the mode is “rule table creation” mode, the production rule creation device 1 inputs the bilingual corpus 150 to the control means 12 via the input / output means 10 (step 1 S6) The phrase pair extraction means 125 extracts the phrase pair from the parallel corpus 150 (step S7: phrase pair extraction step). The extracted phrase pair 112 is stored in the storage unit 11.

  Subsequently, the production rule creation device 1 adds a restriction that the symbol string of the translation destination language on the right side of the production rule of the synchronous context free grammar starts from the terminal symbol based on the phrase pair 112 by the production rule creation unit 126. A generation rule is created. At this time, a hierarchical feature is extracted from the word correspondence 111 by the hierarchical feature extraction means 128 (step S8: generation rule creation step, hierarchical feature extraction step). The created rule 113 and the extracted hierarchical feature 119 are respectively stored in the storage unit 11. And the production | generation rule preparation apparatus 1 matches each translation score calculated from each production | generation rule of the rule 113 with each production | generation rule by the translation score calculation means 127 (step S9: translation score calculation step). The associated generation rule and translation score are stored in the storage unit 11 as the rule table 114.

  If the result of determination in step S 1 is that the mode is “language model creation” mode, the production rule creation device 1 inputs the translation language corpus 140 to the control means 12 via the input / output means 10. (Step S10) A language model is created by the language model learning means 122 (Step S11). The created language model 115 is stored in the storage unit 11. Then, the generation rule creation device 1 extracts the translation destination language bigram feature 116 from the read translation destination language corpus 140 by the language model learning unit 122 (step S12). The extracted translation destination bigram feature 116 is stored in the storage means 11. The translation language bigram feature 116 is extracted when a language model is created.

  The production rule creation device 1 can also be realized by executing a production rule creation program that causes a general computer to execute each of the steps described above. This program can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

According to the generation rule creation device 1 according to the reference example of the present invention, a generation rule is created by adding a restriction that a symbol string of a translation language on the right side starts with a terminal symbol to a generation rule of a synchronous context free grammar. In addition to calculating the translation probability corresponding to this generation rule as a feature, it is possible to extract a binary feature such as a hierarchical feature or a translated language insertion feature based on the generation rule. Therefore, by performing statistical machine translation using the generation rules and features obtained by the generation rule creation device 1, it becomes possible to generate a translation sentence in a manner that is likely to be developed as correct translation.

[Configuration of machine translation device]
FIG. 7 is a functional block diagram showing the configuration of the machine translation apparatus according to the embodiment of the present invention.
The machine translation device 2 defines a probability of correspondence between at least one of the above-described hierarchical features and the above-described translation destination language insertion features, and a word string in the translation source language and a word string in the translation destination language. Using the model, the input word string in the source language is mechanically translated into a word string in the target language corresponding to the input. In the present embodiment, the machine translation device 2 uses the rule table created by the generation rule creation device 1 (see FIG. 1) as a translation model. The machine translation device 2 sequentially creates a new partial hypothesis longer than a predetermined partial hypothesis as a translation target language word string corresponding to the input, which is a translation result of the input translation source language word string. A hypothesis that is a partial hypothesis finally generated by extending a predetermined partial hypothesis is output. The machine translation apparatus 2 includes, for example, a CPU, a RAM, a ROM, an HDD, an input / output interface, and the like. As shown in FIG. 7, the input / output means 20, the storage means 21, the control means 22, and the like. It has.

The input / output means 20 inputs a word string in the source language from the input device K to the control means 22, and outputs a word string in the target language as a translation result from the control means 22 to the output device D. It is. The input / output unit 20 inputs the feature weight learning parallel corpus 250 to the feature weight learning unit 221.
The feature weight learning bilingual corpus 250 is prepared separately from the bilingual corpus 150 used when the generation rule creation device 1 (see FIG. 1) creates a generation rule.

  The storage unit 21 includes, for example, a RAM used for arithmetic processing by the CPU, and a ROM and HDD for storing predetermined programs, various databases, processing results, and the like. For example, the storage means 21 stores a feature weight 211, word information 212, a rule with word range 213, a partial hypothesis 214, and a partial hypothesis score 215 as processing results. Further, the storage means 21 includes a rule table 114, a language model 115, a translation destination language bigram feature 116, a word pair feature 117, a translation destination language, each created by the production rule creation device 1 (see FIG. 1). An insertion feature 118 and a hierarchical feature 119 are stored.

  The control unit 22 includes a feature weight learning unit 221, a word information extraction unit 222, and a translation control unit 223.

  The feature weight learning means 221 includes a feature weight learning parallel translation corpus 250, a translation model stored in the rule table 114, a language model 115, a translation destination language bigram feature 116, a word pair feature 117, and a translation destination language insertion. A weight corresponding to each feature is learned from the feature 118 and the hierarchical feature 119 by an online margin maximization learning method to be described later, and the learning result is stored in the storage unit 21 as a feature weight 211.

  The word information extraction unit 222 divides the sentence of the translation source language input from the input device K via the input / output unit 20 into units of words, and obtains information (word information) about the words constituting the sentence of the translation source language. To extract. The word information includes, for example, a word string, a word position, the number of words, and the like. The extracted word information 212 is stored in the storage means 21. If the sentence of the translation source language input from the input device K has already been divided into words, the word information extraction unit 222 can be omitted.

  The translation control means 223 obtains a hypothesis that covers the entire sentence while expanding the partial hypothesis according to the procedure described later, and finds the optimum (actually semi-optimal) one of them. The generation rule search means 241 and the word A generation rule generation unit with range 242, a partial hypothesis score calculation unit 243, and a hypothesis search unit 244 are provided.

  The generation rule search means 241 searches the rule table 114 for a generation rule applicable for extending a predetermined partial hypothesis.

  The generation rule generation unit with word range 242 searches the generation rule searched by the generation rule search unit 241 based on the number of words and the word position of the words constituting the input word string of the translation source language. A word range indicating the range of the word string of the translation source language covered by the non-terminal symbol of the generated generation rule is added to generate applicable generation rules with word ranges.

Here, the conversion target part (the range covered by the non-terminal symbol) of the input source language word string is expressed as [l, r] at the left end (left) and right end (right) of the word position. I decided to. In the initial stage, for example, if the input is 11 words, [l, r] = [1,11]. In this case, the word range (untranslated word range) of each non-terminal symbol on the source language side of the generation rule is expressed as [l ₁ , r ₁ ], [l ₂ , r ₂ ],. For example, [l ₁ , r ₁ ] = [1, 2]. If there are two non-terminal symbols on the translation source language side of a certain generation rule, two word ranges are also set. In addition, there are a plurality of possibilities as the value of “l ₁ ” or “r ₁ ” for a non-terminal symbol on the translation source language side of a generation rule.

The partial hypothesis score calculation means 243 creates the translated word and the word range on the translation target language side included in the generation rule with word range as a new partial hypothesis, and applies the applicable generation rule from the top down. In addition, the partial hypothesis score indicating the evaluation value of the created partial hypothesis H ′ by extending the new partial hypothesis in the order in which the non-terminal symbols on the translation target language side are arranged from the beginning to the end of the sentence in the applicable generation rule Is calculated.
In the present embodiment, the partial hypothesis score calculation means 243 includes the translation model stored in the rule table 114, the translation destination language bigram feature 116, the word pair feature 117, the translation destination language insertion feature 118, and the hierarchical feature 119. Is calculated as a partial hypothesis score S (H ′).

  Specifically, the partial hypothesis score calculation means 243 includes a combination of “a word string from the beginning of a sentence in the translation target language” and “a stack that holds a range of untranslated words in the word string in the translation source language”. Based on the partial hypothesis H, a new partial hypothesis H ′ is generated using the applicable generation rule with word range. Creating H ′ based on H is called extension of the partial hypothesis. As the partial hypothesis is expanded, the word string from the beginning of the translated language in the partial hypothesis is added sequentially from the beginning of the sentence to the end of the sentence. Further, the partial hypothesis score calculating means 243 saves memory. Share word strings with other partial hypotheses, and if the number of words in the source language is J, the initial partial hypothesis (initial value of the partial hypothesis) is “empty string” and “[1, J] is the only stack.

Partial hypothesis score calculating means 243, and the number of words that are in the source language portion hypothesis H 'translated and m (0 ≦ m ≦ J) , the priority queue Q _0, Q _1, ..., a Q _J Enter a partial hypothesis H ′. That is, the partial hypothesis score calculation means 243 starts with Q ₀ = {initial partial hypothesis} and expands the partial hypothesis stored in the priority-added queue Q _m in synchronization with the number of translated words in the translation source language. .

When expanding a partial hypothesis, the partial hypothesis score calculation means 243 pops the untranslated word range [l, r] from the top of the stack (from the top of the stack). Generation rules corresponding to input sentences in the source language are managed using the Earley chart structure. By using the chart structure, it is possible to efficiently find a generation rule applicable to the partial hypothesis. For example, when the partial hypothesis score calculation means 243 extracts a partial hypothesis from a generation rule with a word range as shown in the equation (29) generated from the equation (17), the word range [l ₂ , r ₂ ], [l ₁ , r ₁ ] are pushed onto the stack in order, so that [l ₁ , r ₁ ] is processed first. In this way, the translation target language side is guaranteed to always generate a translation from the beginning of the sentence.

  The partial hypothesis score calculation means 243 adds the difference score to the score S (H) of the original partial hypothesis H, as shown in the equation (30), thereby increasing the score S of the expanded partial hypothesis H ′. (H ′) is calculated.

と表記する。ここで、生成された翻訳先言語の単語列は、「ｉ」番目の単語から「ｉ＋ε」番目の単語で構成されている。 Here, for example, the features shown in Table 5 are used as h _m (r). In addition, when generating a hypothesis, the word string of the target language newly generated by the expansion of the partial hypothesis,

Is written. Here, the generated word string of the translation destination language is composed of the “i + ε” -th word from the “i” -th word.

Here, P _{e | f} (r), P _{f | e} (r), Lex _{e | f} (r), and Lex _{f | e} (r) are generation rules defined in the rule table 114 of the storage unit 21. The feature corresponding to r. Further, the feature weight λ _m in the above equation (30) is assumed to be defined in advance in the feature weight 211 stored in the storage unit 21. The last four features (h ₇ (r) to h ₁ (r)) shown in Table 5 are binary features and generally have higher-order elemental properties. Note that “i”, “j”, “k”, and “l” described in Table 5 indicate integers determined by the number of binary features.

In the present embodiment, from the above equation (30) and Table 5, the partial hypothesis score calculation means 243 finally calculates the score S (H ′) of the partial hypothesis H ′ using the equation (31). Further, the partial hypothesis score calculation means 243 holds only the maximum Y (for example, 1000) partial hypotheses in the priority queues Q ₀ , Q ₁ ,..., Q _J. The partial hypothesis score calculation unit 243, partial hypothesis score S (H ') is the priority queue Q _0, Q _1, ..., if less than the product of a constant with the largest part hypothesis score in Q _J Discard the partial hypothesis H ′. Thereby, the partial hypothesis score calculation means 243 can perform effective pruning on each partial hypothesis stored in the priority-added queues Q ₀ , Q ₁ ,..., Q _J.

The hypothesis search means 244 searches for a predetermined partial hypothesis that can be applied to the input word string of the source language, and expands the predetermined partial hypothesis so that the entire sentence of the source language finally generated Among the partial hypotheses corresponding to (referred to as a hypothesis), a hypothesis having the maximum partial hypothesis score is searched. Specifically, the hypothesis search means 244 searches the hypothesis H ^ having the maximum hypothesis score from the relationship of the expression (32) among a predetermined number of hypotheses obtained from the entire sentence of the translation source language. Since the hypothesis H shown in the equation (32) is the partial hypothesis H included in the priority queue Q _J , the hypothesis H ^ obtained by the equation (32) maximizes the value of the partial hypothesis score S (H). This can be realized by obtaining a partial hypothesis H (that is, hypothesis H).

  Further, the hypothesis searching means 244 outputs a word string from the beginning of the translation language corresponding to the obtained hypothesis to the end of the sentence as a translation result to the output device D via the input / output means 20.

<Online margin maximization learning method>
Here, online margin maximization learning (Online Large-Margin Training) performed by the feature weight learning means 221 will be described. The feature weight learning means 221 executes the online learning algorithm shown in FIG. This online learning algorithm is different from a general online learning algorithm in that quasi-correct answer data that can be generated by a decoder (hypothesis searching means 244) is utilized.

Best _k shown in line number "3" (·) is, by the decoder, the learning samples (f ^t, e ^t) by using the weight vector w ⁱ from k-best list C ^t (k number of the target language sentence data) Here is a function that generates Here, each learning sample is intended for a single source language sentence f ^t for certain t, it has a plurality of target language sentence e ^t. Accordingly, “e ^t ” represents a vector indicating a plurality of reference translations.

The oracle _m (•) indicated by the line number “4” merges the k-best list C ^t and the previous quasi-correct answer data O ^t to generate (update) m quasi-correct answer data. . The correct answer data is determined through a process described later using the newly generated quasi-correct answer data O ^t .

As shown in the row number “5”, the previous weight vector w ⁱ includes the k-best list C ^t generated by the row number “3”, the quasi-correct answer data O ^t generated by the row number “4”, and Is used to generate a new weight vector w ^{i + 1} . Wrt is an abbreviation for with respect to. The feature weight learning means 221 repeats the process from the line number “3” to the line number “6” T times over the learning sample ( ^ft , ^et ) as an inner loop, and further, the line number “2”. To the line number “7” is repeated N times as an outer loop, and the average value of the weight vectors w ⁱ is returned as indicated by the line number “9”.

In the present embodiment, the feature weight learning unit 221 obtains an optimum weight vector w ⁱ by using online margin maximization learning using Expression (33) in the process of the row number “5”. For online margin maximization methods, see “Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer, Online passive-aggressive algorithms, Journal of Machine Learning Research, 7: 551-585, March 2006 ( Hereinafter referred to as a reference).

Loss function in equation (34), based on the reference translation e ^t, is a function that calculates the difference between e ^ and e '. As the loss function, for example, an index such as BLEU, which is an index based on an n-gram accuracy rate, can be used. For BLEU, see “Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu, BLEU: a method for automatic evaluation of machine translation, In Proc. Of ACL 2002, p. 311-318, Philadelphia, Pennsylvania, 2002” It is described in.

The feature weight learning means 221 forms a margin between a plurality of correct translations (e ^) and a plurality of erroneous candidates (e ') output from the decoder, as shown in the equation (34). The margin is made larger than the value of the loss function. Generally, in online margin maximization learning, there is only one correct answer data (see, for example, the above-mentioned reference). However, since the correct answer cannot be uniquely determined in translation, the feature weight learning unit 221 uses a plurality of correct answer data (quasi-correct answer data). Furthermore, since the true correct answer (e ^) cannot be defined depending on the loss function L used in the online margin maximization learning, the feature weight learning unit 221 changes the k-best list C ^t output from the decoder to the correct answer (e ^). The closest one is used as correct data (quasi-correct data O ^t ). The optimum weight vector w ^{i + 1} indicating the left side of the above equation (33) can be derived, for example, by quadratic programming.

  With this method (the algorithm shown in FIG. 8), the feature weight learning unit 221 learns toward a plurality of correct answers, and can learn the optimum model parameters even for a small amount of learning data. It is. Moreover, since the weighting for that is determined by the quadratic programming method, according to this method, it is possible to design a flexible constraint. For example, by adding a loss function such as NIST in addition to a loss function such as BLEU, it is possible to add a constraint that the NIST is optimized simultaneously with the correct BLEU answer. NIST is described in “George Doddington, Automatic evaluation of machine translation quality using n-gram co-occurrence statistics, In Proc. ARPA Workshop on Human Language Technology, 2002”.

  The feature weight learning unit 221, the word information extraction unit 222, and the translation control unit 223 (generation rule search unit 241, generation rule generation unit with word range 242, partial hypothesis score calculation unit 243, and hypothesis search unit 244). The CPU is realized by developing a predetermined program stored in the HDD or the like of the storage unit 21 in the RAM and executing it.

[Operation of machine translation device]
The operation of the machine translation apparatus 2 shown in FIG. 7 will be described with reference to FIG. 9 (refer to FIG. 7 as appropriate). FIG. 9 is a flowchart showing the operation of the machine translation apparatus shown in FIG. In advance, the machine translation device 2 uses the feature weight learning unit 221 to perform the feature weight learning parallel translation corpus 250, the rule table 114, the language model 115, the translation destination language bigram feature 116, the word pair feature 117, and the translation destination. Based on the language insertion feature 118 and the hierarchical feature 119, the weight of the value of the feature function is learned, and the feature weight 211 as a learning result is stored in the storage means 21 (step S21: feature weight learning step). .

  Then, the machine translation device 2 inputs the sentence in the source language input from the input device K via the input / output unit 20 to the word information extraction unit 222 (step S22). The machine translation device 2 divides the inputted sentence of the source language (input sentence) into words by the word information extraction unit 222, and extracts the word string, the number of words J, and the word position of each word (step S23). ). The extracted word string, word string word count J, and word position are stored in the storage unit 21 as word information 212.

The machine translation device 2 creates an initial partial hypothesis H ₀ by the hypothesis search means 244, empties _J priority queues Q ₀ ,..., Q _J according to the number of input words J, of which The initial partial hypothesis H ₀ is stored in the priority queue Q ₀ (step S24). Then, the machine translation apparatus 2 initializes the partial hypothesis score S (H ₀ ) for the initial partial hypothesis H ₀ and the identification variable m of the priority queue with the hypothesis search means 244. That is, S (H ₀ ) = 0 and m = 0 are set (step S25). Subsequently, the machine translation device 2 sequentially pops each applicable partial hypothesis H stored therein from the m-th priority queue Q _m by the generation rule search means 241, and each partial hypothesis. Each applicable generation rule r that can expand H is searched from the rule table 114. Then, the number of translated words (number of translated words) V (r) described on the translation source language side of each generation rule r as a search result is acquired, and the identification variable m of the current priority queue. Is added to this value, the value of the number n of translated words for the partial hypothesis H to be processed is updated. That is, n = m + V (r) is set (step S26: generation rule search step).

Then, the machine translation device 2 generates each generation rule r ′ with word range applicable to the searched generation rule by the generation rule generation unit 242 with word range (step S27: generation rule generation with word range). Step). Then, the machine translation apparatus 2 creates partial hypotheses H ′ by expanding the partial hypotheses H to be processed by the generation rules r ′ with word ranges by the partial hypothesis score calculating means 243, respectively. A partial hypothesis score S (H ′) is calculated for H ′ based on the equation (31) described above. Then, the created partial hypothesis H ′ is stored in the n-th priority queue Q _n and predetermined pruning is performed (step S 28: partial hypothesis score calculation step). Here, the partial hypothesis H ′ that has become unnecessary due to pruning is deleted from the priority queue Q _n . The calculated partial hypothesis score S (H ′) is stored in the partial hypothesis score 215 of the storage means 21, but the partial hypothesis score S (H ′) for the partial hypothesis H ′ deleted by pruning is deleted. The Rukoto.

  Then, the machine translation apparatus 2 determines whether or not all applicable generation rules r ′ with word ranges have been selected by the hypothesis search means 244 (step S29). If applicable r ′ still exists (step S29: No), the machine translation device 2 returns to step S27. On the other hand, if all r ′ are selected (step S29: Yes), the hypothesis searching means 244 determines whether all applicable generation rules r have been selected (step S30). If applicable r still exists (step S30: No), the machine translation device 2 returns to step S26. On the other hand, if all r are selected (step S30: Yes), the hypothesis searching unit 244 determines whether all applicable partial hypotheses H have been selected (step S31). If applicable H still exists (step S31: No), the machine translation device 2 returns to step S26. On the other hand, if all H are selected (step S31: Yes), the hypothesis searching means 244 determines whether or not the value of the identification variable m of the current priority queue is equal to the number of input words J (m = J). (Step S32). If m ≠ J (step S32: No), the machine translation apparatus 2 increments the identification variable m of the priority-added queue by the hypothesis search means 244. That is, m = m + 1 is set (step S33). Then, it returns to step S26.

On the other hand, if m = J (step S32: Yes), the J-th priority queue Q _J stores a plurality of partial hypotheses H covering the entire sentence of the source language. These partial hypotheses H can be regarded as hypotheses H for sentences in the source language. Similarly, the partial hypothesis score S (H) for the partial hypothesis H stored in the priority queue Q _J is referred to as a hypothesis score S (H) in this sense. In this case, the machine translation device 2 searches the hypothesis search means 244 for the hypothesis H having the maximum hypothesis score S (H) from the J-th priority queue Q _J (step S34: hypothesis search). Step). Then, the machine translation device 2 outputs the searched hypothesis as a sentence in the translation destination language by the hypothesis search means 244 (step S35). As a result, a word string from the beginning of the translation target language to the end of the sentence corresponding to the searched hypothesis is output to the output device D.

  The machine translation apparatus 2 can also be realized by executing a machine translation program that causes a general computer to execute the above steps. This program can be distributed via a communication line, or can be written on a recording medium such as a CD-ROM for distribution.

[Concrete example]
Specific examples will be described with reference to FIG. 10, Table 6, Table 7, and Table 8.
FIG. 10 is a diagram illustrating an extension example from the partial hypothesis illustrated in FIG. 7 to the hypothesis. Table 6 shows a translation source language sentence composed of 11 words, and Table 7 shows a generation rule applicable to the translation source language sentence shown in Table 6. In addition, the “type of generation rule” in Table 7 indicates which of the above formulas (13) to (17) corresponds. Table 8 shows the order of application of the generation rules shown in Table 7 and the generation rules with word ranges based thereon.

  In the initial state, the partial hypothesis score calculation means 243 empties the stack and, as shown in FIG. 10, the word range [1, 11] covering the entire word string (sentence) of the translation source language in the state “0”. To push. In the state “0”, since the partial hypothesis H ′ (0) is the initial partial hypothesis, it is a set of the word range [1, 11] pushed onto the stack and an empty string. The partial hypothesis score S (H ′) of the partial hypothesis H ′ (0) is “0”.

Next, in the state “1”, the partial hypothesis score calculation unit 243 pops the range [1, 11] from the stack, and the generation rule search unit 241 generates a generation rule applicable to the popped range from Table 7. Select r (3). Since the word position in the input sentence of “ha” corresponding to the translation solution “The” in the generation rule r (3) is “3”, the generation rule generation unit with word range 242 generates the generation rule as shown in Table 8. A generation rule r ′ (3) with a word range such that the word ranges of the non-terminal symbol X ₍₁₎ and the non-terminal symbol X ₍₂ ) in r (3) are [1, 2] and [4, 11], respectively. Generate. In this generation rule r ′ (3) with word range, the non-terminal symbol X ₍₁₎ must be processed before the non-terminal symbol X ₍₂₎ on the translation destination language side. Therefore, as shown in FIG. 10, the partial hypothesis score calculation means 243 pushes the word range [4, 11] corresponding to the nonterminal symbol X ₍₂₎ onto the stack in the generation rule with word range r ′ (3). After that, the word range [1, 2] corresponding to the non-terminal symbol X ₍₁₎ is pushed. The partial hypothesis score calculation means 243 converts the word ranges [1, 2] and [4, 11] pushed onto the stack and the translation solution “The” as a word string from the head of the translation target language into the partial hypothesis H ′ ( 1). The partial hypothesis score calculation means 243 calculates a partial hypothesis score for the partial hypothesis H ′ (1).

Next, in the state “2”, the partial hypothesis score calculation unit 243 pops the word range [1, 2] from the stack, and the generation rule search unit 241 generates a generation rule applicable to this word range from Table 7. Select r (1). Since the word position in the input sentence “international” corresponding to the translation solution “international” in the generation rule r (1) is “1”, the generation rule generation unit with word range 242 generates the generation rule as shown in Table 8. word range of r (1) non-terminal symbol X in ₍₁₎ to produce a conditioned word range generation rule r '(1) such that [2,2]. As shown in FIG. 10, the partial hypothesis score calculation means 243 pushes the word range [2, 2] on the stack in the generation rule r ′ (1) with word range. The partial hypothesis score calculation means 243 translates the word range [2, 2] pushed onto the stack, the previously pushed word range [4, 11], and the translation solution “The international” as a word string from the beginning of the target language. Is a partial hypothesis H ′ (2). The partial hypothesis score calculation means 243 calculates a partial hypothesis score for the partial hypothesis H ′ (2).

  Next, in the state “3”, the partial hypothesis score calculating unit 243 pops the word range [2, 2] from the stack, and the generation rule searching unit 241 uses the generation rule applicable to this word range from Table 7. Select r (2). In the generation rule r (2), the translation solution “terrorism” corresponding to “terrorism” is described, but since there is no non-terminal symbol, the generation rule generation unit with word range 242 uses the generation rule r (2) as it is. The generation rule r ′ (2) with word range is used. The partial hypothesis score calculation means 243 does not operate the stack because the word range is not specified in the generation rule r ′ (2) with word range. As shown in FIG. 10, the partial hypothesis score calculation means 243 partially converts the word range [4, 11] previously pushed onto the stack and the translation solution “The international terrorism” as a word string from the head of the translated language. Let it be hypothesis H ′ (3). The partial hypothesis score calculation means 243 calculates a partial hypothesis score for the partial hypothesis H ′ (3).

In the same manner, when an operation corresponding to the state “4” to the state “9” is performed in the application order described in Table 8, the stack becomes empty, and the partial hypothesis score calculation unit 243 develops the partial hypothesis. To generate a hypothesis. At this time, the translation solution as a word string from the sentence head of the translation target language is a word string of 10 words as follows.
"The international terrorism also is a possible threat in Japan"

  Here, since the generation rule r (9) has two terminal symbols (words) “is a”, the partial hypothesis score calculation means 243 translates 10 words in nine state transitions. To do. FIG. 10 shows that the translation target language side was generated from the beginning of the sentence to the end of the sentence in the process of partial hypothesis development.

  FIG. 11 is a diagram showing a tree structure of the synchronous context free grammar corresponding to the extension example to the hypothesis shown in FIG. This is a sequence of generation rules used for partial hypothesis expansion expressed as a tree. In FIG. 11, (1) to (9) represent the expansion order of the generation rules of the synchronous context free grammar. The generation rules start with a terminal symbol on the destination language on the right side, and the tree is expanded from top to bottom in the order from the beginning of the target language to the end of the target language. Generated automatically. In contrast, the source language is not always analyzed from the beginning to the end of the sentence.

Table 9 shows an example different from the generation rule with word range shown in Table 8. Table 9 is the same as Table 8 except that the contents of the application orders “1” and “2” are different. In this case, the generation rule r (1) is selected in the state “1”, and the generation rule r (3) is selected in the state “2”. At this time, since the translation solution as a word string from the head of the target language is as follows, the hypothesis score of the hypothesis in the case of Table 8 is as follows. Smaller than.
"International The terrorism also is a possible threat in Japan"

  According to the machine translation apparatus 2 of the present embodiment, in addition to the translation model for each generation rule stored in the rule table 114, the weight corresponding to the feature including binary features such as the hierarchical feature 119 and the translated language insertion feature 118 Is calculated based on the feature weight learning parallel corpus 250, a partial hypothesis score is calculated based on the feature vector and the feature weight vector, and a hypothesis having a maximum partial hypothesis score (hypothesis score in this case) is calculated. Search as a word string in the target language. Therefore, it is possible to generate a translation sentence in a manner that is likely to be developed as correct translation. As a result, it is possible to improve the translation accuracy of the word string in the target language.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. For example, in the present embodiment, the Japanese-English translation from Japanese to English has been described as an example, but the combination of languages is not limited to this. If a production rule is created by the production rule creation device, this translation device can be used between any multilinguals.

Further, the production rule creation device 1 according to the reference example of the present invention is configured to include both the hierarchical feature extraction unit 128 and the translation destination language insertion feature extraction unit 132, but the hierarchical feature extraction unit 128 and the translation destination Only one of the language insertion feature extraction means 132 may be provided. In this case, the same effect can be obtained.
In the present embodiment, the machine translation device 2 is configured to perform statistical machine translation including both the hierarchical feature 119 and the translation target language insertion feature 118, but the hierarchical feature 119 and the translation target language Only one of the insertion features 118 may be included. In this case, the same effect can be obtained. Furthermore, although the machine translation device 2 uses the rule table created by the generation rule creation device 1 (see FIG. 1) as the translation model, the translation model to be used is not limited to the rule table. .

In order to confirm the effect of the present invention, a translation experiment from Arabic to English was performed by setting the translation source language to “Arabic” and the translation destination language to “English”.
Specifically, as the bilingual corpus 150 (see FIG. 1), an “Arabic / English corpus” distributed by LDC (Linguistic Data Consortium) was used. This bilingual corpus (training set) 150 consists of approximately 3.8 million sentences.
In addition, MT2003 evaluation set (663 sentence: development set) is used as the bilingual corpus 250 for feature weight learning (see FIG. 7), and MT2004 (707 sentence: open test set) and MT2005 (1,056 sentence: open test set) are used for the test. ) Was used.
Further, “English Gigaword” distributed from LDC was used as the translation language corpus 140 (see FIG. 1). That is, “English Gigaword” was used for learning the language model 115 and extracting the translation language bigram feature 116.

<Experiment on word normalization>
As an experiment on word normalization, we conducted an experiment comparing features that assumed the surface form of words and features assumed to normalize words.
(Example 1) Only the surface form of the word was assumed.
Example 2 Prefix / suffix normalization was assumed in addition to the word surface form.
Example 3 In addition to the surface form of words, word class normalization was assumed.
Example 4 In addition to the surface form of words, digit normalization was assumed.
(Example 5) In addition to the surface form of words, normalization of prefix and suffix, normalization of word classes, and normalization of numbers (all token types) Assumed.

Common experimental conditions for Examples 1 to 5 are as follows.
The feature weight learning unit 221 sets the number of repetitions in online margin maximization learning to 50 (N = 50).
For each learning sample, the hypothesis searching means 244 (decoder) outputs 1000-best, from which the top 10 translations and 10 correct answer data are extracted.
As a loss function, BLEU was used and evaluated by both BLEU and NIST.
Table 10 shows the experimental results in this case.

  Comparing Example 1 and Example 2, with respect to the MT2003 evaluation set (denoted as 2003 (dev)), Example 2 has significantly improved performance compared to Example 1, but MT2004 (2004 ) And MT2005 (2005), the performance was degraded. Therefore, Example 2 causes overlearning. On the other hand, when Example 1 was compared with Example 3, the performance of Example 3 was improved compared to Example 1 without causing overlearning. Similarly, the performance of Example 4 and Example 5 was improved as compared to Example 1 without overlearning.

<Experiment regarding combinations of binary features>
Experiments were performed to compare combinations of binary features after using the features of Example 5 assuming total normalization.
Example 6 Only word pair features were used.
(Example 7) In addition to the feature (word pairs) of Example 6, the translated language bigram feature (target bigram) was used.
(Example 8) In addition to the features of Example 7 (word pairs, target bigram), the translation language insertion feature (insertion) was used.
Example 9 In addition to the features of Example 8 (word pairs, target bigram, insertion), word-based hierarchical features were used.
Table 11 shows the experimental results in this case.

  When Example 6 and Example 7 are compared, Example 7 has caused overlearning. On the other hand, when Example 6 and Example 8 are compared, the performance of Example 8 is improved compared to Example 6 without overlearning. Similarly, the performance of Example 9 also improved compared to Example 6 without causing overlearning.

<Experiment regarding existence of binary features>
As in Example 9, an experiment was performed comparing four types of features (with binary features) with no binary features (no binary features). Specifically, a two-fold cross validation method was used for a set combining MT2003 evaluation set, MT2004 and MT2005 (total set). That is, half of the total set was used for the feature weight learning corpus and the other half was used as a test set, and this was alternately performed to obtain an average. Here, Example 10 (online) uses four types of features. Moreover, the conventional method described in the nonpatent literature 4 was used as a comparative example (baseline) which does not use a binary feature. Table 12 shows the experimental results in this case. As shown in Table 12, the performance of Example 10 was significantly improved compared to the comparative example.

It is a block diagram which shows the structure of the production | generation rule production apparatus which concerns on the reference example of this invention. It is a figure which shows the example of word correspondence of a Japanese-English parallel translation. It is a figure which shows the example of the rule table shown in FIG. It is a figure which shows the example of the word pair feature shown in FIG. It is a figure which shows the example of the hierarchical feature shown in FIG. It is a flowchart which shows operation | movement of the production | generation rule preparation apparatus shown in FIG. It is a block diagram which shows the structure of the machine translation apparatus which concerns on embodiment of this invention. It is a figure which shows an online learning algorithm. It is a flowchart which shows operation | movement of the machine translation apparatus shown in FIG. It is a figure which shows the example of an extension from the partial hypothesis shown in FIG. 7 to a hypothesis. It is a figure which shows the tree structure of the synchronous context free grammar corresponding to the example extended to the hypothesis shown in FIG. It is a figure which shows the tree structure of the example of mistranslation. It is a figure which shows the tree structure of the example of correct translation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Generation rule creation apparatus 2 Machine translation apparatus 10 Input / output means 11 Storage means 111 Word correspondence 112 Phrase pair 113 Rule 114 Rule table 115 Language model 116 Destination language bigram feature 117 Word pair feature 118 Destination language insertion feature 119 Hierarchical feature DESCRIPTION OF SYMBOLS 12 Control means 121 Mode determination means 122 Language model learning means 123 Word correspondence creation means 124 Inter-language correspondence feature extraction means 125 Phrase pair extraction means 126 Generation rule creation means 127 Translation score calculation means 128 Hierarchical feature extraction means 131 Word pair feature extraction Means 132 Translation destination language insertion feature extraction means 140 Translation destination language corpus 150 Bilingual corpus 20 Input / output means 21 Storage means 211 Feature weight 212 Word information 213 Rule with word range 214 Partial hypothesis 215 parts Hypothesis score 22 controller 221 feature weight learning means 222 word information extracting unit 241 generates rule searching unit 242 words scoped generation rule generation means 243 portion hypothesis score calculation unit 244 hypothesis search means 250 feature weight learning corpus

Claims (4)

A hierarchical feature in which a binary feature is defined for each word pair defined from upper and lower relations on a syntax tree constituting a word string of a translation source language or a word string of a translation destination language in parallel translation learning data, and the translation A word inserted in the word string of the translation target language when the word corresponding to the word constituting the word string of the translation target language in the learning data is not included in the word string of the translation source language, and the translation source language And at least one of the translation target language insertion features represented by one of the binary values and the other of the binary values for each word pair included in the word sequence, Translation corresponding to the input, which is the translation result of the input source language word string, using a translation model that defines the likelihood of correspondence between the source language word string and the target language word string Predetermined partial hypotheses as the target language word string A machine translation apparatus for outputting hypothesis is finally generated partial hypotheses by expanding the predetermined partial hypotheses are sequentially create a longer new partial hypotheses than Rasore,
A weight corresponding to a feature including at least one of the hierarchical feature and the translation target language insertion feature and the translation model is learned based on feature weight learning parallel translation learning data, and the learning result is used as a feature weight. Feature weight learning means for storing in the storage means;
Evaluation of the created partial hypothesis is an inner product of a feature vector including at least one of the hierarchical feature and the translation target language insertion feature and the translation model as elements, and a weight vector indicating the feature weight. A partial hypothesis score calculating means for calculating a partial hypothesis score indicating a value;
Among the partial hypotheses finally generated by searching for a predetermined partial hypothesis applicable to the input word string of the source language and extending the predetermined partial hypothesis, the partial hypothesis score A machine translation device comprising: a hypothesis search means for searching for a partial hypothesis having a maximum value as the hypothesis. A hierarchical feature in which a binary feature is defined for each word pair defined from upper and lower relations on a syntax tree constituting a word string of a translation source language or a word string of a translation destination language in parallel translation learning data, and the translation A word inserted in the word string of the translation target language when the word corresponding to the word constituting the word string of the translation target language in the learning data is not included in the word string of the translation source language, and the translation source language And at least one of the translation target language insertion features represented by one of the binary values and the other of the binary values for each word pair included in the word sequence, Translation corresponding to the input, which is the translation result of the input source language word string, using a translation model that defines the likelihood of correspondence between the source language word string and the target language word string Predetermined partial hypotheses as the target language word string A machine translation method of machine translation apparatus which outputs the hypothesis is finally generated partial hypotheses by sequentially creating a longer new partial hypotheses expanding the predetermined partial hypotheses than Rasore,
A feature weight learning unit learns a weight corresponding to a feature including at least one of the hierarchical feature and the translation target language insertion feature and the translation model based on feature weight learning parallel translation learning data, A feature weight learning step of storing a learning result as a feature weight in a storage means;
A partial hypothesis score calculating means calculates an inner product of a feature vector including at least one of the hierarchical feature and the translation target language insertion feature and the translation model as an element, and a weight vector indicating the feature weight, A partial hypothesis score calculating step for calculating as a partial hypothesis score indicating an evaluation value of the created partial hypothesis;
Of the partial hypotheses finally generated by searching for a predetermined partial hypothesis applicable to the input source language word string by the hypothesis search means and expanding the predetermined partial hypothesis And a hypothesis search step of searching for a partial hypothesis having the maximum partial hypothesis score as the hypothesis. A machine translation program for causing a computer to execute the machine translation method according to claim 2. A computer-readable recording medium on which the machine translation program according to claim 3 is recorded.

Images