JP2006099208A - Statistical machine translation apparatus and statistical machine translation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus capable of performing translation with higher accuracy in a statistical machine translation apparatus using a parallel translation phrase.
A Japanese-English machine translation decoder includes a Japanese phrase N-gram model 56, an English phrase N-gram model 58, an English language model 60, an English-to-Japanese phrase translation model 68, and Japanese input. A segmentation processing unit 80 that performs all possible segmentations on the sentence 40, and a phrase sequence in which English phrases are arranged with probability in any order using the models 68, 56, 58, and 60 in accordance with the obtained segmentation. And a A * search processing unit 88 that searches for and outputs the top M routes with the highest probability among the lattices created by the lattice creation unit 84.
[Selection] Figure 3

Description

  The present invention relates to a machine translation apparatus, and more particularly, to a machine translation apparatus that efficiently translates using a parallel translation of a phrase that appears in a parallel corpus in statistical machine translation. In the present specification, “phrase” means a group of one or a plurality of consecutive words (word string).

  From the standpoint of statistically performing machine translation, as proposed in Non-Patent Document 1, the problem of translating the source language sentence f into the target language e sentence is formulated as the next maximization problem. It becomes.

この問題に関し、雑音を含む通信路モデルを適用すると次の式が得られる。

When the channel model including noise is applied to this problem, the following equation is obtained.

統計機械翻訳においてフレーズを用いた手法を試みた文献として、式（２）の第１番目の因子Ｐ（ｆ｜ｅ）を、さらに制約を課した一連のフレーズの翻訳を掛合わせたもので近似しているものがある（非特許文献２、３、４）。

Approximating the first factor P (f | e) of equation (2) by multiplying a series of restricted phrase translations as a reference for using a phrase-based technique in statistical machine translation (Non-Patent Documents 2, 3, and 4).

ただし/ｆ_i（本明細書では、符号の直前の「/」は、直後の符号の直上に記載されるべき上線をあらわすものとする。）は、文ｆに対するフレーズセグメンテーション（フレーズ分割）がされたフレーズセグメンテーション後の文/ｆのｉ番目のフレーズのことをいい、ａ_iはフレーズセグメンテーションされたテキストのｉ番目のフレーズアライメントのことをいう。
Ｐ．Ｆ．ブラウン他、「機械翻訳に対する統計的アプローチ」、計算機言語学、第１６巻、第２号、ｐｐ．７９−８５、１９９０年（P. F. Brown et al., "A statistical approach to machine translation," Computational Linguistics, vol. 16, no. 2, pp. 79-85, 1990.） Ｐ．ケーン他、「フレーズによる統計翻訳」、ＨＬＴ−ＮＡＡＣＬ２００３予稿集、ｐｐ．４８−５４、２００３年（P. Koehn et al., "Statistical phrase-based translation," in Proc. of HLT-NAACL 2003, pp. 48-54 2003.) Ｓ．フォーゲル他、「ｃｍｕ統計翻訳システム」、ＭＴサミットＩＸ予稿集、ｐｐ．４０２−４０９、２００３年（S. Vogel et al., "The cmu statistical translation system," in Proceedings of MT Summit IX, pp. 402-409, 2003.） Ｃ．ティルマン、「統計機械翻訳のための射影拡張アルゴリズム」、自然言語処理における経験法２００３年大会予稿集、ｐｐ．１−８、２００３年（C. Tillmann, "A projection extension algorithm for statistical machine translation," in Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pp. 1-8, 2003.） Ｈ．ダウメ ＩＩＩ他、「文書／要約アライメントのためのフレーズによるｈｍｍアプローチ」、ＥＭＮＬＰ２００４予稿集、ｐｐ．１１９−１２６、２００４年（H. Daume III et al., "A phrase-based hmm approach to document/abstract alignment," in Proceedings of EMNLP 2004, pp. 119-126, 2004.） Ａ．Ｐ．デンプスター他、「不完全なデータからのｅｍアルゴリズムによる最大尤度」、王室統計学会ジャーナル、第Ｂ巻、第３９号、ｐｐ．１−３８、１９７７年（A. P. Dempster et al., "Maximum likelihood from incomplete data via the em algorithm," Journal of the Royal Statistical Society, vol. B, no. 39, pp. 1-38, 1977.） Ｆ．Ｊ．オク他、「さまざまな統計的アライメントモデルの系統的比較」、コンピュータ言語学、第２９巻、第１号、ｐｐ．１９−５１、２００３年（F. J. Och et al. "A systematic comparison of various statistical alignment models," Computational Linguistics, vol. 29, no. 1, pp. 19-51, 2003） Ｆ．Ｊ．オク他、「統計機械翻訳のための弁別トレーニングおよび最大エントロピーモデル」、ＡＣＬ２００２予稿集、ｐｐ．２９５−３０２、２００２年（F. J. Och et al., "Discriminative training and maximum entropy models for statistical machine translation," in Proc. of ACL 2002, pp. 295-302, 2002.） Ｆ．Ｊ．オク、「統計機械翻訳における最小誤差率学習」、ＡＣＬ２００３予稿集、ｐｐ．１６０−１６７、２００３年（F. J. Och, "Minimum error rate training in statistical machine translation," in Proc. of ACL 2003, pp. 160-167, 2003.） Ｗ．Ｈ．プレス他、「Ｃ＋＋の数値的レシピ」、ケンブリッジ大学出版会、２００２年（W. H. Press et al., "Numerical Recipes in C++," Cambridge University Press, 2002.） Ｎ．ユッフィインク他、「統計機械翻訳における単語グラフの生成」、自然言語処理における経験法大会（ＥＭＮＬＰ０２）予稿集、ｐｐ．１５６−１６３、２００２年（N. Ueffing et al., "Generation of word graphs in statistical machine translation," in Proc. of Conference on Empirical Methods for Natural Language Processing (EMNLP02), pp. 156-163, 2002.）

However, / f _i (in this specification, “/” immediately before the code represents an overline to be described immediately above the code immediately after) is subjected to phrase segmentation (phrase division) for the sentence f. I is the i-th phrase of the sentence / f after the phrase segmentation, and a _i is the i-th phrase alignment of the phrase segmented text.
P. F. Brown et al., “Statistical Approach to Machine Translation”, Computer Linguistics, Vol. 16, No. 2, pp. 79-85, 1990 (PF Brown et al., "A statistical approach to machine translation," Computational Linguistics, vol. 16, no. 2, pp. 79-85, 1990.) P. Kane et al., “Statistical Translation by Phrases”, HLT-NAACL2003 Proceedings, pp. 48-54, 2003 (P. Koehn et al., "Statistical phrase-based translation," in Proc. Of HLT-NAACL 2003, pp. 48-54 2003.) S. Vogel et al., “Cmu Statistical Translation System”, MT Summit IX Proceedings, pp. 402-409, 2003 (S. Vogel et al., "The cmu statistical translation system," in Proceedings of MT Summit IX, pp. 402-409, 2003.) C. Tillman, “Projection Extension Algorithm for Statistical Machine Translation”, Experiential Methods in Natural Language Processing 2003 Proceedings, pp. 1-8, 2003 (C. Tillmann, "A projection extension algorithm for statistical machine translation," in Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pp. 1-8, 2003.) H. Daume III et al., “Phrase hmm approach for document / summary alignment”, EMNLP 2004 proceedings, pp. 119-126, 2004 (H. Daume III et al., "A phrase-based hmm approach to document / abstract alignment," in Proceedings of EMNLP 2004, pp. 119-126, 2004.) A. P. Dempster et al., “Maximum likelihood by em algorithm from incomplete data”, Journal of Royal Statistical Society, Vol. 1-38, 1977 (AP Dempster et al., "Maximum likelihood from incomplete data via the em algorithm," Journal of the Royal Statistical Society, vol. B, no. 39, pp. 1-38, 1977.) F. J. et al. Ok et al., “Systematic Comparison of Various Statistical Alignment Models”, Computer Linguistics, Vol. 29, No. 1, pp. 19-51, 2003 (FJ Och et al. "A systematic comparison of various statistical alignment models," Computational Linguistics, vol. 29, no. 1, pp. 19-51, 2003) F. J. et al. Oku et al., “Discrimination Training and Maximum Entropy Model for Statistical Machine Translation”, ACL 2002 Proceedings, pp. 295-302, 2002 (FJ Och et al., "Discriminative training and maximum entropy models for statistical machine translation," in Proc. Of ACL 2002, pp. 295-302, 2002.) F. J. et al. Ok, “Minimum Error Rate Learning in Statistical Machine Translation”, ACL 2003 Proceedings, pp. 160-167, 2003 (FJ Och, "Minimum error rate training in statistical machine translation," in Proc. Of ACL 2003, pp. 160-167, 2003.) W. H. Press et al., "C ++ Numerical Recipe", Cambridge University Press, 2002 (WH Press et al., "Numerical Recipes in C ++," Cambridge University Press, 2002.) N. Yufiink et al., “Generation of word graphs in statistical machine translation”, Proceedings of Empirical Convention on Natural Language Processing (EMNLP02), pp. 156-163, 2002 (N. Ueffing et al., "Generation of word graphs in statistical machine translation," in Proc. Of Conference on Empirical Methods for Natural Language Processing (EMNLP02), pp. 156-163, 2002.)

However, in the conventional method, the method for obtaining the phrase translation pair is simple. Even if P (/ f _i | / e _ai ) is estimated according to the equation (3), for example, the correlation between phrases, etc. Since the condition for is not included, there is a problem that the accuracy of translation is lowered.

  Therefore, an object of the present invention is to provide a machine translation device that can perform translation with higher accuracy in a statistical machine translation device that uses parallel phrases.

  Another object of the present invention is to provide a machine capable of performing translation with higher accuracy by coordinating estimation of phrase segmentation and estimation of phrase translation in a statistical machine translation apparatus using parallel translation phrases. It is to provide a translation device.

  A machine translation device according to a first aspect of the present invention is a translation device that translates an input sentence in a source language into a target language, the phrase N-gram model in the source language, the phrase N-gram model in the target language, and a predetermined language A storage means for storing a model and a translation model by a phrase from the target language to the source language, a segmentation means for performing segmentation on the input sentence of the source language, and a storage means in accordance with the segmentation obtained by the segmentation means Source language phrases N-gram model, target language phrases N-gram model and language model, and target language-to-source language phrase translation model, and target language phrases in any order, Phrase lattices are created by placing the probabilities obtained by using the phrase N-gram model of one source language, the phrase N-gram model and language model of the target language, and the translation model by phrase, with each node and edge aligned. And a search means for searching for the top M phrase sequences giving the maximum probability among the lattices created by the lattice creation means.

  When an input sentence in a source language is given using an unprecedented concept called a phrase translation model, the top M translation sentences having the highest statistical probability of translation for the input sentence can be output. Since the probability not only at the word level but also at the phrase level is used, it is possible to perform translation with high accuracy by directly using the correspondence between phrases between the input sentence and the translated sentence.

  Preferably, the search means includes means for searching the top M phrase sequences giving the maximum probability among the lattices created by the lattice creation means using the A * search algorithm.

  More preferably, the lattice creation means includes means for creating a lattice using only nodes satisfying a predetermined probability condition.

  The phrase-based translation model may be formed by a combination of a target language phrase N-gram model, a source language phrase segmentation model, and a target language-to-source language phrase translation model.

  The statistical machine translation program according to the second aspect of the present invention is a statistical machine translation program that, when executed by a computer, causes the computer to operate as one of the machine translation devices described above.

[Basic concept]
In the conventional technique, the method for obtaining a translation pair and the estimation of P (/ f _i | / e _ai ) in Equation (3) are different problems, and therefore, there is a problem that improvement in accuracy cannot be expected. It is thought that there was. In the present embodiment, machine translation using phrases is performed based on the following idea. That is, instead of the equation (3) used in the prior art, new variables / f and / e are introduced, and P (f | e) is calculated by the following equation (4): The relationship corresponding to phrase translation is directly reflected in the translation.

ただし/ｆは入力文ｆのセグメンテーションを表し、/ｅはターゲット文ｅのセグメンテーションを表す。

However, / f represents the segmentation of the input sentence f, and / e represents the segmentation of the target sentence e.

  In equation (4), P (f, / f, / e | e) is further decomposed into three terms.

式（５）の第１項はセグメンテーションされた入力文/ｆが再構成により入力文ｆとなる確率を表す。第２項は/ｅと/ｆという二つのフレーズシーケンスに対する翻訳確率を表す。最後の項は、フレーズセグメンテーションされた文/ｅが文ｅから生成される尤度を示す。本明細書では、これらの項をそれぞれフレーズセグメンテーションモデル、フレーズ翻訳モデル，およびフレーズＮグラムモデルと呼ぶ。以下、これらにつき個別に説明する。

The first term of Equation (5) represents the probability that the segmented input sentence / f becomes the input sentence f by reconstruction. The second term represents the translation probabilities for the two phrase sequences / e and / f. The last term indicates the likelihood that a phrase segmented sentence / e is generated from sentence e. In this specification, these terms are referred to as a phrase segmentation model, a phrase translation model, and a phrase N-gram model, respectively. These will be described individually below.

-Phrase N-gram model-
The phrase N-gram model can be approximated as follows.

式（６）において、Ｐ（/ｅ_i｜/ｅ_i-1）は、互いに隣接する、翻訳されたフレーズに関するバイグラム制約として取扱う。

In Equation (6), P (/ e _i | / e _i-1 ) is treated as a bigram constraint on translated phrases that are adjacent to each other.

The phrase N-gram model can be easily calculated using a forward / backward algorithm by extending all possible phrase segmentations for sentence e to a lattice structure / E as shown in FIG. As shown in FIG. 1, each node in the lattice represents a specific phrase / E _i (i = 1 to 6) in the sentence e, and these nodes / E ₁ to / E ₆ are connected to each other by edges. It is. Further, each edge is given a probability that a phrase corresponding to the node at the front end of the edge and a phrase corresponding to the node at the rear end are successively generated. For example, the node / E _i and the immediately preceding node / E _{i ′} are connected by an edge with a probability P (/ E _i | / E _{i ′} ). In the example of FIG. 1, the probability P (/ E ₂ | / E ₁ ) is attached to the edge connecting the node / E ₂ and the node immediately preceding it (/ E _{1 in} FIG. ₁ ).

The value of P (/ E _i | / E _{i ′} ) is estimated by the following procedure.

  1) Initialize the probability table with some uniform value.

2) For each sentence e in the learning corpus, the forward / backward algorithm is used to estimate the posterior probability P (/ E _i , / E _{i ′} | e) in the lattice.

  3) Using the estimated posterior probability as the word occurrence frequency, the prior probability is calculated using the maximum likelihood algorithm as follows.

ステップ２および３を所定の終了条件が成立するまで繰返す。終了条件としては、予め定められた繰返し回数を完了したことなどの条件を用いることができる。

Steps 2 and 3 are repeated until a predetermined end condition is satisfied. As the termination condition, a condition such as completion of a predetermined number of repetitions can be used.

-Phrase segmentation model-
According to the modeling shown in the equation (4), P (f | / f, / e, e) can be obtained as a source sentence f by rearranging the phrase segmented sentence / f, Can be regarded as a distortion probability. In the present embodiment, instead of this, this model is regarded as representing the likelihood that a specific phrase segment / f _j will occur in the sentence f. That is,

このセグメンテーションは、前節に述べたフレーズＮグラムモデルのユニグラム事後確率として実現することができる。このユニグラム事後確率は、文ｆに対するラッティス構造/Ｆを用い、前向き・後向きアルゴリズムを用いて効率的に計算できる。

This segmentation can be realized as a unigram posterior probability of the phrase N-gram model described in the previous section. This unigram posterior probability can be efficiently calculated using the forward / backward algorithm using the lattice structure / F for the sentence f.

このフレーズセグメンテーションモデルは、ソース文が与えられたときに、あるフレーズに対してある重みを与えるものと考えることができる。仮にフレーズ長を１に限定すると、すなわちどのフレーズも単語を一つしか含まないものとすると、フレーズセグメンテーションの結果は一通りとなる。したがってフレーズセグメンテーションモデルはどのフレーズにも「１」という重みを割当てる。

This phrase segmentation model can be considered to give a certain weight to a certain phrase when a source sentence is given. If the phrase length is limited to 1, that is, if every phrase contains only one word, the results of phrase segmentation will be one. Therefore, the phrase segmentation model assigns a weight of “1” to any phrase.

-Phrase translation model-
The phrase translation model is approximated so that the phrase translation is obtained as a product of the translation of each phrase.

ただしａ_iは単語アライメントによる翻訳モデル（例えばＩＢＭ（登録商標）モデル）において見られるようなフレーズアライメントを表す。

Here, a _i represents a phrase alignment as found in a translation model based on word alignment (for example, an IBM (registered trademark) model).

  As one implementation form of the phrase translation model, one in which all frame translation pairs appearing in the learning corpus are stored in a table form together with their appearance probabilities.

-HMM statistical translation by phrase-
When all of the above phrase N-gram model, phrase segmentation model, and phrase translation model are integrated, Equation (4) can be rewritten as follows.

フレーズセグメンテーションがされた文/ｅと文/ｆとを、それぞれ対応する/Ｅおよび/Ｆというラッティス構造にまで拡張すると、式（１２）は隠れマルコフモデル（ＨＭＭ）と考えることができる。ただしこの場合、図２に示すように、ラッティス/Ｆ内の各ソースフレーズ/Ｆ_j（ｊ＝１〜６）は、ラッティス/Ｅ内のターゲットフレーズである状態/Ｅ_i（ｉ＝１〜６）から出力された観測量として扱われる。図２に示す例でいえば、ラッティス/Ｆ内のフレーズ/Ｆ₂は、ラッティス/Ｅ中のフレーズ/Ｅ₃からの観測量として扱われる。

When the phrase segmented sentence / e and sentence / f are expanded to corresponding lattice structures of / E and / F, respectively, Expression (12) can be considered as a hidden Markov model (HMM). However, in this case, as shown in FIG. 2, each source phrase / F _j (j = 1 to 6) in the lattice / F is a target phrase in the lattice / E / E _i (i = 1 to 6). ) Is treated as the observation amount output. In the example shown in FIG. 2, the phrase / F ₂ in the lattice / F is treated as an observation amount from the phrase / E ₃ in the lattice / E.

In FIG. 2, the phrase / F ₂ generated in the lattice / F is observed with the probability P (/ F ₂ | / E ₃ ) from the phrase / E ₃ . Lattice / the probability that the next phrase / E ₃ followed by phrases / E ₁ P in _{E (/ E 3 | / E} 1), the probability that phrases / F ₂ is generated in the segmentation of the sentence f P (/ F ₂ | f ), The values of the terms corresponding to i = 3 and j = 2 in equation (12) are obtained by multiplying them. By calculating this for all combinations of nodes (phrases) in Lattice / E and Lattice / F and adding them up, the value of Equation (4) can be approximated.

  Non-Patent Document 5 has already proposed a problem that the phrase-based HMM structure associates a document with a summary. In the method described in Non-Patent Document 5, the jump probability is clearly encoded as a state transition, and it can be said that the probability substantially corresponds to the alignment probability in the statistical translation model using words. The use of jumps or alignment probabilities in a clear way can help complete translation modeling, but instead the phrase-based HMM structure has the problem of requiring a vast search space for learning. is there.

  In the method according to this embodiment, the state transition uses a phrase N-gram model and a phrase bigram connection probability, but ignores the phrase alignment probability. Accordingly, it can be said that the phrase-based HMM translation model is an incomplete model. However, because of its simplicity, parameter estimation can be performed at high speed.

-Parameter estimation-
As described above, the parameters for the phrase-based HMM translation model can be efficiently estimated using the forward / backward algorithm.

In the process of the forward / backward algorithm, two auxiliary variables α (e _i1 ⁱ² , f _j1 ^j2 ) and β (e _i1 ⁱ² , f _j1 ^j2 ) are defined. α (e _i1 ⁱ² , f _j1 ^j2 ) represents a forward estimate of the probability that the phrase e _i1 ⁱ² is translated into f _j1 ^j2 after output of all the combinations of phrases present in e ₁ ^i1-1 . Similarly, β (e _i1 ⁱ² , f _j1 ^j2 ) is a backward estimate of the probability that the phrase e _i1 ⁱ² is translated into the phrase f _j1 ^j2 when all the combinations of phrases on the right side of e _{i2 + 1} ^l are considered. Value.

  Therefore, the forward / backward algorithm is formulated in the form of solving the following recurrence formula.

ＥＭアルゴリズムでよく見られるような局所的収束という問題（非特許文献６）を避けるため、本実施の形態では、フレーズによる翻訳モデルの学習にあたって、初期値として非特許文献７に記載のＧＩＺＡ＋＋学習より得られるレキシコンモデルを用いる。さらに、フレーズＮグラムモデルとフレーズセグメンテーションモデルとを単言語コーパスを用いて別々に学習させ、ＨＭＭ学習の繰返し時には固定させておく。

In order to avoid the problem of local convergence (Non-Patent Document 6) often seen in the EM algorithm, in this embodiment, the initial value of GIZA ++ learning described in Non-Patent Document 7 is used for learning a translation model using phrases. Use the resulting lexicon model. Furthermore, the phrase N-gram model and the phrase segmentation model are separately learned using a monolingual corpus, and are fixed when the HMM learning is repeated.

-Phrase segment induction-
Equations (13) and (14) include the summation process for all possible paths, respectively, on the left and right sides of the lattice structure / E, and the summation process for all possible segmentations for / F. Yes. Since this processing still requires enormous calculation even if dynamic programming is used, the segmentation to be considered is limited in advance in this embodiment.

  In deriving a phrase pair, first, the product of two phrase translation probabilities shown in the following equation is used for all the bilingual phrase pairs that can be found in the bilingual learning corpus.

ただしｃｏｕｎｔ（/ｅ，/ｆ）は二つのフレーズ/ｅおよび/ｆの共起頻度である。式（１５）の基本的な概念は、２方向の対応関係を見ることにより、バイリンガルのフレーズの対応をとらえる、ということである。

However, count (/ e, / f) is the co-occurrence frequency of two phrases / e and / f. The basic concept of equation (15) is that the correspondence between bilingual phrases can be grasped by looking at the correspondence in two directions.

  Furthermore, for other phrases, the product set / union of Viterbi word alignment using the two-way model P (e | f) and P (f | e) calculated by GIZA ++ (Non-patent Document 2). Based on exhaustive derivation.

  After the phrase translation pairs are extracted, monolingual phrase lexicons for them are extracted and used as possible segmentation for the source and target sentences.

-Decoder-
The decision rule for calculating the best translation is performed by the logarithmic linear sum of all the components of the translation model shown in Non-Patent Document 8.

ただしＰｒ_j（ｅ，ｆ）は翻訳モデルの構成要素であって、例えばフレーズＮグラムモデルまたは言語モデルなどであり、λ_jは各モデルに与えられた重みである。重みλ_jはＩＩＳ（Improved Iterative Scaling）またはＧＩＳ（Generalized Iterative Scaling）アルゴリズムを用いた最尤基準を用いても、滑降シンプレックス法（Downhill Simplex Method：非特許文献９）のように制約なしの最適化アルゴリズムを用いた最小誤差率基準（非特許文献９）を用いてもよい。

However, Pr _j (e, f) is a component of the translation model, for example, a phrase N-gram model or a language model, and λ _j is a weight given to each model. The weight λ _j is an unconstrained optimization as in the Downhill Simplex Method (Non-Patent Document 9), even if the maximum likelihood criterion using IIS (Improved Iterative Scaling) or GIS (Generalized Iterative Scaling) algorithm is used. You may use the minimum error rate standard (nonpatent literature 9) using an algorithm.

  The decoder is similar to a decoder based on a word graph (Non-patent Document 11) and incorporates a complicated structure of a sub model so that coding can be performed in multi-pass. In the first pass of decoding, a beam search is used to generate a translation word graph, or lattice, for the input sentence. In this pass, all sub-models of the phrase-based HMM translation model are used with a trigram language model and a class 5 gram language model with words in the target language. In the next pass, the N best path of translation is searched for this word graph using the A * search method.

[Constitution]
FIG. 3 shows a block diagram of the statistical machine translation system 20 according to the present embodiment. This statistical machine translation system 20 is a Japanese-English translation device that translates a Japanese sentence (f) into an English sentence (e). That is, in this embodiment, the source language f is Japanese and the target language e is English.

  Referring to FIG. 3, the statistical machine translation system 20 includes a Japanese monolingual corpus (hereinafter referred to as “Japanese corpus”) 30 and an English monolingual corpus (hereinafter referred to as “English corpus”) 34. Japanese and English parallel corpus 32, Japanese phrase N-gram model 56, English phrase N-gram model 58, English language model 60, and phrase translation model from Japanese corpus 30, parallel corpus 32, and English corpus 34 68, a Japanese phrase N-gram model 56, an English phrase N-gram model 58, an English language model 60, and a phrase created by the phrase-based HMM translation model creation section 36. Using the translation model 68, the Japanese input sentence 40 is translated into English And a decoder (statistical machine translation apparatus) 42 for outputting the translated sentence 44. In the statistical machine translation system 20 shown in FIG. 3, a phrase bigram is used as the phrase Ngram. The English phrase N-gram model 58, the Japanese phrase N-gram model 56, and the phrase translation model 68 constitute an HMM translation model 38 using phrases.

  The phrase-based HMM translation model creation unit 36 creates a Japanese phrase N-gram model creation unit 50 for creating a Japanese phrase N-gram model 56 from the Japanese corpus 30 and an English phrase N-gram model 58 from the English corpus 34. An English phrase N-gram model creating unit 52 for creating the English language model 60 from the English corpus 34. Although not shown, the English language model 60 includes an English word trigram language model and an English class 5 gram language model as described above. The class N-gram language model is a word N-gram language model that focuses on the part of speech, and is an N-gram language model that is created by regarding the same part of speech as the same word.

  It should be noted that the Japanese phrase N-gram model creation unit 50 adopts only a predetermined number of N-grams appearing in the Japanese corpus 30 and creates the bilingual corpus 32 in creating the Japanese phrase N-gram model 56. The restriction is that only Japanese N-grams appearing as phrase pairs should be adopted. Similarly, the English phrase N-gram model creation unit 52 adopts only a predetermined upper number of N-grams appearing in the English corpus 34 when creating the English phrase N-gram model 58, and phrase pairs appearing in the parallel corpus 32. This is in accordance with the restriction that only English N-grams appearing as

  The phrase-based HMM translation model creation unit 36 further includes a phrase segmentation model creation unit 62 for creating a Japanese phrase segmentation model 66 based on the Japanese phrase N-gram model 56, and an English phrase N-gram model. 58 and a phrase segmentation model 66, and an HMM learning unit 70 that estimates the parameters of the HMM that constitutes the phrase translation model 68.

  On the other hand, when the Japanese input sentence 40 is given, the decoder 42 stores all of the segmentation processing unit 80 for performing all possible segmentations on the input sentence 40 and the segmentation created by the segmentation processing unit 80. And a lattice creation unit 84 for creating a lattice comprising translation phrases for the input sentence 40 based on the entire segmentation of the input sentence 40 stored in the segmentation storage unit 82. The lattice creation unit 84 uses the phrase translation model 68 to replace each Japanese phrase stored in the segmentation storage unit 82 with the corresponding English phrase when creating the lattice, and further converts the English phrase into an arbitrary phrase. Lattices are created by combining them in order, and the Japanese phrase N-gram model 56, English phrase N-gram model 58, and English language model 60 are used in addition to assign a probability to each edge and node of the lattice. Note that since the search space is very wide when creating lattices using English phrases, the lattice creation unit 84 performs a beam search using the above-described probabilities, and creates lattices while leaving only edges and nodes with large probabilities.

  The decoder 42 further stores a lattice storage unit 86 for storing lattices composed of English phrases with probabilities created by the lattice creation unit 84 and an English lattice with probabilities stored in the lattice storage unit 86. * By performing a search, an A * search processing unit 88 is included for outputting M pieces of high scores among lattice paths as M best translations 44.

FIG. 4 shows a procedure for estimating the value of P (/ E _i | / E _{i ′} ) in creating the English phrase N-gram model 58. Referring to FIG. 4, first, in step 100, a probability table for storing the above probabilities is initialized with some uniform value. In step 102, the repeated control variable i is initialized to zero. In step 104, 1 is added to the variable i. In step 106, it is determined whether or not the value of the variable i exceeds 2. If the value of the variable i exceeds 2, the process is terminated. If the value of the variable i is 2 or less, the process proceeds to step 108.

  In step 108, the processing from step 110 to step 116 is repeated for each sentence in the English corpus 34 (see FIG. 3).

First, in step 110, all possible segmentation is performed on the sentence in the corpus, and a lattice is created based on the result. At step 112, probabilities are assigned to all edges in the lattice as shown in FIG. In step 114, the posterior probability P (/ E _i , / E _{i ′} | e) of each node is estimated by the forward / backward algorithm.

  In step 116, it is determined whether or not the above processing has been completed for all sentences. If it has not been completed, the processing from step 110 is repeated. After completion for all sentences, in step 118, the posterior probability estimated in step 114 is used as the word occurrence frequency, and the prior probability of the phrase N-gram model is expressed by equation (7) using the maximum likelihood algorithm. calculate.

  Then, it returns to step 104 again and performs the next repetition. In the present embodiment, the processing ends when the above processing is repeated twice.

  FIG. 5 shows a procedure for calculating a phrase segmentation model for a Japanese sentence F. Referring to FIG. 5, first, in step 140, all possible phrase segmentations are obtained for sentence F, and a lattice is created. In step 142, the forward probability α and the backward probability β are calculated by the forward / backward algorithm using the Japanese phrase N-gram model 56 for all phrases appearing in the lattice obtained in step 140. In step 144, P (/ F | F) is calculated according to equation (10). This process is performed for all sentences.

  FIG. 6 shows a procedure for calculating a phrase translation model. Referring to FIG. 6, first, in step 160, lattice pairs <F, E> are generated in units of phrases for all parallel translations in the parallel corpus 32 shown in FIG.

  In step 162, the processing from step 164 to step 170 is repeated. First, in step 164, the posterior probability P (/ F | F) for each segmentation / F of the sentence F is obtained by the lattice of the source sentence F. In step 166, the forward probability α and the backward probability β for the combination of the segmentation / E and the segmentation / F in the lattice of the target sentence E are calculated using the forward / backward algorithm. In step 168, the count is added as the probabilities of the phrases included in the segmentation / E and the segmentation / F.

  In step 170, it is determined whether or not the processing in steps 164 to 168 described above has been performed for all <F, E>. If not yet performed, the processing is repeated from step 164 again.

  When the processing is completed for all <F, E>, in step 172, P (/ F | / E) is calculated based on the count value calculated in step 168, and the processing ends.

[Operation]
The operation of the statistical machine translation system 20 shown in FIG. The operation of the statistical machine translation system 20 has two phases as is apparent from the above description. The first is a phase in which a phrase translation model 68 and related Japanese phrase N-gram model 56, English phrase N-gram model 58, and English language model 60 are created. Second, using the phrase translation model 68, the Japanese phrase N-gram model 56, the English phrase N-gram model 58, and the English language model 60 thus created, the input sentence 40 is translated and the translated sentence 44 is output. It is. This will be described in order below.

-Model creation phase-
Assume that a Japanese corpus 30, a bilingual corpus 32, and an English corpus 34 are prepared in advance. The Japanese phrase N-gram model creation unit 50 creates a Japanese phrase N-gram model 56 from the Japanese corpus 30. The English phrase N-gram model creation unit 52 creates an English phrase N-gram model 58 from the English corpus 34. The English language model creation unit 54 creates an English language model 60 from the English corpus 34.

  The phrase segmentation model creation unit 62 creates a phrase segmentation model 66 based on the Japanese phrase N-gram model 56 and the Japanese corpus 30.

  The HMM learning unit 70 uses the phrase segmentation model 66 and the English phrase N-gram model 58 thus created, and further learns the phrase translation model 68 using the parallel corpus 32.

-Translation phase-
When the phrase-based HMM translation model 38, that is, the Japanese phrase N-gram model 56, the English phrase N-gram model 58, the phrase translation model 68, and the English language model 60 are prepared, the decoder 42 can translate the input sentence 40. . The segmentation processing unit 80 performs all possible segmentations on the input sentence 40 and stores them in the segmentation storage unit 82. Based on the segmentation stored in the segmentation storage unit 82, the lattice creation unit 84 creates a lattice of a translation sentence having a phrase as a node, using the phrase translation model 68 and the like. The probability calculated based on the model is assigned to the lattice edges and nodes of the translation sentence created at this time. The created lattice is stored in the lattice storage unit 86.

  The A * search processing unit 88 uses the A * search algorithm to search for the M best path with the highest probability among the lattices with probability stored in the lattice storage unit 86, and translates the corresponding phrase sequence into them. 44 is output.

[effect]
As described above, in the present embodiment, the correspondence relationship of phrases included in the bilingual corpus can be directly modeled and converted into an HMM, and translation phrase pairs can be efficiently created and translated. There is an effect that the accuracy is higher than the conventional one.

[Realization by computer]
The statistical machine translation system 20 of this embodiment is realized by computer hardware, a program executed by the computer hardware, and data stored in the computer hardware. FIG. 7 shows the external appearance of the computer system 330, and FIG. 8 shows the internal configuration of the computer system 330.

  Referring to FIG. 7, the computer system 330 includes a computer 340 having an FD (flexible disk) drive 352 and a CD-ROM (compact disk read only memory) drive 350, a keyboard 346, a mouse 348, and a monitor 342. including.

  Referring to FIG. 8, in addition to FD drive 352 and CD-ROM drive 350, computer 340 includes CPU (central processing unit) 356 and bus 366 connected to CPU 356, FD drive 352 and CD-ROM drive 350. And a read only memory (ROM) 358 for storing a boot-up program and the like, and a random access memory (RAM) 360 connected to the bus 366 for storing a program command, a system program, work data, and the like. Computer system 330 further includes a printer 344.

  Although not shown here, the computer 340 may further include a network adapter board that provides a connection to a local area network (LAN).

  A computer program for causing the computer system 330 to operate as the statistical machine translation system 20 is stored in the CD-ROM 362 or FD 364 inserted in the CD-ROM drive 350 or FD drive 352 and further transferred to the hard disk 354. . Alternatively, the program may be transmitted to the computer 340 through a network (not shown) and stored in the hard disk 354. The program is loaded into the RAM 360 when executed. The program may be loaded directly into the RAM 360 from the CD-ROM 362, from the FD 364, or via a network.

  In this embodiment, the phrase translation model 68, the Japanese phrase N-gram model 56, the English phrase N-gram model 58, and the English language model 60 shown in FIG. 3 are all stored in the hard disk 354 shown in FIG. The data is read out to the RAM 360 as necessary and used when executing the program. The segmentation storage unit 82, the lattice storage unit 86, and the like shown in FIG.

  This program includes a plurality of instructions for causing the computer 340 to operate as the statistical machine translation system 20 according to this embodiment. Some of the basic functions required to perform this operation are provided by operating system (OS) or third party programs running on the computer 340 or various toolkit modules installed on the computer 340. Therefore, this program does not necessarily include all functions necessary for realizing the system of this embodiment. This program includes only instructions that execute the above-described operation as the statistical machine translation system 20 by calling an appropriate function or “tool” in a controlled manner so as to obtain a desired result. Just go out. The operation of computer system 330 is well known and will not be repeated here.

[Modification]
In the embodiment described above, both the Japanese phrase N-gram model 56 and the English language model 60 are created from the independent monolingual Japanese corpus 30 and English corpus 34. In general, since the content of a monolingual corpus is substantial, the accuracy of the Japanese phrase N-gram model 56, the English phrase N-gram model 58, and the English language model 60 created by these procedures is high. However, the present invention is not limited to such an embodiment. For example, when the number of parallel translations included in the parallel corpus 32 is sufficiently large, a Japanese phrase N-gram model 56 is created from a Japanese sentence in the parallel translation corpus 32, and an English phrase N-gram model 58 and an English language model are created from the English sentence. 60 may be created.

  In the present embodiment, the process shown in FIG. 4 is repeated twice, and the process shown in FIG. 6 ends only once, but the present invention is not limited to such an embodiment. For example, the processing in FIG. 4 may be repeated one or more times and an arbitrary number of times. In the process shown in FIG. 6, the process may return to step 160 after step 172, and the repetition may be performed twice or more. Furthermore, instead of fixing the number of repetitions, the repetition may be terminated when some condition is satisfied.

  When the Japanese phrase N-gram model 56 and the English phrase N-gram model 58 are created by the Japanese phrase N-gram model creation unit 50 and the English phrase N-gram model creation unit 52, in the above embodiment, the Japanese corpus 30, A bilingual corpus 32 and an English corpus 34 are used as constraints. However, the present invention is not limited to such an embodiment. For example, in addition to or instead of this, a Japanese phrase N-gram model 56 and an English phrase N-gram model 58 may be created using a phrase-type bilingual dictionary as a constraint.

  In the embodiment described above, translation from Japanese to English is performed. However, the present invention is not limited to such an embodiment. The present invention can be applied to any combination of languages as long as an appropriate corpus is available.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a figure which shows an example of the phrase lattice structure of a target language. It is a figure which shows the relationship of the phrase lattice structure of the source language used at the time of phrase translation model creation, and a target language. 1 is a block diagram of a statistical machine translation system 20 according to an embodiment of the present invention. It is a flowchart which shows the estimation procedure of the value of P (/ _Ei | / _{Ei '} ) in creation of the English phrase N-gram model 58. It is a flowchart which shows the procedure which calculates the phrase segmentation model with respect to the Japanese sentence F. FIG. It is a flowchart which shows the procedure which calculates a phrase translation model. 1 is an external view of a computer system that realizes an embodiment of the present invention. FIG. 8 is a block diagram of the computer system shown in FIG. 7.

Explanation of symbols

  20 statistical machine translation systems, 30 Japanese corpus, 32 parallel corpus, 34 English corpus, 36 phrase HMM translation model generator, 38 phrase HMM translation model, 40 input sentences, 42 decoder, 44 translations, 50 Japanese phrases N Gram model creation unit, 52 English phrase N-gram model creation unit, 54 English language model creation unit, 56 Japanese phrase N-gram model, 58 English phrase N-gram model, 60 English language model, 62 Phrase segmentation model creation unit, 66 phrase Segmentation model, 68 phrase translation model, 70 HMM learning unit, 80 segmentation processing unit, 82 segmentation storage unit, 84 lattice creation unit, 86 lattice storage unit

Claims (5)

A translation device that translates an input sentence in a source language into a target language,
For storing a phrase N-gram model of the source language (N is an integer of 2 or more), a phrase N-gram model of the target language and a predetermined language model, and a translation model by a phrase from the target language to the source language Storage means;
Segmentation means for performing segmentation on the source language input sentence;
According to the segmentation obtained by the segmentation means, the phrase N-gram model of the source language stored in the storage means, the phrase N-gram model and the language model of the target language, and from the target language to the source language The target language phrases in any order, and the source language phrase N-gram model, the target language phrase N-gram model and the language model, and the phrase Lattice creation means for creating a lattice lattice with the probability obtained using the translation model attached to each node and edge,
A statistical machine translation apparatus comprising: search means for searching for top M phrase sequences giving the maximum probability among the lattices created by the lattice creation means. 2. The statistics according to claim 1, wherein the search means includes means for searching, using the A * search algorithm, the top M phrase sequences giving the maximum probability among the lattices created by the lattice creation means. Machine translation device. The statistical machine translation device according to claim 1, wherein the lattice creation unit includes a unit for creating the lattice using only a node satisfying a predetermined probability condition. The phrase translation model is formed by a combination of the phrase N-gram model of the target language, a phrase segmentation model of the source language, and a phrase translation model from the target language to the source language. Item 4. The statistical machine translation device according to any one of Items 3 to 4. A statistical machine translation program that, when executed by a computer, causes the computer to operate as the statistical machine translation apparatus according to any one of claims 1 to 4.

Images