JP5288371B2 - Statistical machine translation system - Google Patents

Description

  The present invention relates to statistical machine translation (SMT), and more particularly to improvement of class-dependent SMT.

It is known that topic-dependent modeling is effective in improving the quality of a model in speech recognition. Recently, experiments in the field of machine translation (prior art non-patent documents 1, 2 and 3) have shown that class-specific models are also useful for translation. According to Non-Patent Document 1, topic dependency is predicted by classifying a source sentence by a classifier that divides the data into a set before starting the decoding process and then learning by all of the source sentences in the preprocessing pass. This is accomplished by independently decoding these sets using separate models specific to the given class.
Hirofumi Yamamoto et al., 2007. A bilingual cluster-based model for statistical machine translation. EMNLP-CoNLL-2007 (Natural Computer Learning Joint Meeting following ACL 2007, Experimental Method Conference at Natural Language Processing Competition), Prague, Czech Republic, pp. 514-523.

(Hirofumi Yamamoto et al. 2007. Bilingual cluster based models for statistical machine translation. EMNLP-CoNLL-2007 (Conference on Empirical Methods in Natural Language Processing Conference on Computational Natural Language Learning Joint Meeting following ACL 2007), Prague, Czech Republic; pp 514-523.)
Andrew Finch et al., 2007. Speech translation system for NICT / ATR IWSLT2007. IWSLT 2007, Trento, Italy.

(Andrew Finch et al. 2007. The NICT / ATR speech translation system for IWSLT 2007. IWSLT 2007, Trento, Italy.)
George Foster and Roland Kuhn, 2007. Adaptation of mixed models for SMT. Proceedings of the second workshop on statistical machine translation, ACL, pages 128-135, Prague, Czech Republic.

  (George Foster and Roland Kuhn. 2007. Mixture-model adaptation for SMT. In Proceedings of the Second Workshop on Statistical Machine Translation, ACL, pp. 128-135, Prague, Czech Republic.)

  Topic-dependent or class-dependent modeling improves machine translation accuracy. However, the accuracy greatly depends on the accuracy of the classifier. If the input sentence is classified into an incorrect topic or class, the accuracy of translation is greatly deteriorated.

  Accordingly, one of the objects of the present invention is to provide an SMT apparatus that can more stably and robustly translate an input sentence of a specific class.

  Another object of the present invention is to provide an SMT apparatus capable of more stably and robustly translating a specific class of input sentences.

  The statistical machine translation apparatus according to the first aspect of the present invention includes means for determining a vector of probabilities representing class membership of a source sentence. The vector element represents the probability that the probability of the source sentence belongs to one of a set of predetermined classes. The apparatus further includes a plurality of class specific statistical sub-decoders provided for each class of the predetermined set of classes. The decoder is statistically trained with a respective set of training data for each class. Each of the decoders outputs the probability of the translated word or word sequence in the target language for each word or word sequence in the source sentence. The apparatus further includes means for estimating the most likely translation hypothesis in the target language of the source sentence according to the probability of a possible word sequence in the target language. The probability of possible word sequences in the target language is calculated by interpolating the probabilities output by the plurality of subdecoders for each word or word sequence in the target language according to a probability vector.

  The means for determining class membership determines a probability vector. The vector element represents the probability that the source sentence belongs to each class. The plurality of statistical sub-decoders outputs the probability of the translated word or word sequence in the target language for each word or word sequence in the source sentence. The estimation means estimates the most likely translation hypothesis according to the word or word sequence probabilities, which are calculated by interpolating the probabilities output by the sub-decoder.

  Preferably, the plurality of classes include a general class and a plurality of specific classes, and the plurality of specific classes are obtained by dividing the general class.

  More preferably, one element of the vector corresponding to the general class is a constant in the range of 0 to 1.

  More preferably, the apparatus further includes normalizing means for normalizing the elements of the vector so that the sum of the elements is 1.

  The means for determining the probability vector may be statistically trained based on a maximum entropy model and assigning membership probabilities to each of the classes.

  Preferably, each of the plurality of class specific statistical sub-decoders calculates a probability according to a class specific language model, a class specific translation model, a class specific length model, or a class specific distortion model, or any combination of these models. To do.

  The approach of the present invention is a generalization of the prior art Non-Patent Document 1 in many respects. The technique of the present invention makes it possible to use a large number of models in the decoding process itself. Each contribution of the set of class specific models is dynamically controlled during decoding by a set of interpolation weights, as described below. These weights can be changed for each sentence. In the previous approach, the interpolation weight is essentially 1 (indicating that the source sentence is the same topic as the model) or 0 (indicating that the source sentence is a different topic), Either.

  One advantage of the present invention is that this is a flexible approach. That is, a source sentence can belong to many classes to various degrees. Here, a probability class representing a class membership was determined using a probability classifier. These probabilities are used directly as blend weights for each class dependent model in the set of interpolated models.

  Another feature of the system of the present invention is that it includes a general model constructed from all data along with a set of class specific models. This results in an accurate and stable translation.

  The approach of this embodiment differs from all previous approaches in terms of class dependent models. Prior to the prior art Non-Patent Document 1, only class-dependent language models were used. Both Non-Patent Documents 1 and 3 extend this to include a translation model. In the inventive approach, all models that can include distortion and target length models are combined into an SMT system within a single framework.

  A bilingual corpus is a collection of sentence pairs. Each pair includes a first language sentence and a second language sentence. Each sentence is a translation of the other. Sentences in a bilingual corpus are segmented into words or phonemes and labeled with part-of-speech labels.

  The language model (LM) gives a word appearance probability on condition that N−1 other words appear before it. The N-gram LM is constructed (trained) with statistics obtained from the target portion of the training set of the bilingual corpus.

  The translation model (TM) gives the probability that a word in the first language is made another word in the second language. In this embodiment, TM is obtained statistically from the training set.

  The length model (LeM) gives a penalty each time a word in the translation (target) is added to the average. The length model is obtained from the target portion of the sentence pair in the training set.

  The distortion model (DM) provides a penalty for the relative distance between two source language phrases associated with two adjacent phrases in the target language. DM is obtained statistically from the training set.

1. Introduction This embodiment weights and combines multiple SMT systems, allowing for probabilistic and flexible weighting between topic-dependent models for all models in the system. This embodiment is an application of this technology and improves the quality of the dialogue system by building and combining class-based models for question sentences and narrative sentences.

  The STM system of this embodiment differs from the previous class-dependent translation method in that the class-dependent format of all models is directly integrated into the decoding process. The system of this embodiment uses a probabilistic blend weight between models, but this weight can be dynamically changed from segment to segment depending on the characteristics of the source segment.

  The system of this embodiment relates to the translation of questions and narratives using a class dependent model. To accomplish this, the system handles the general class of two models of conversational sentences, a set of two models specially constructed to deal with sentences that fit into one of the questions and descriptions. Integrate with a third set built for

  For the purposes of this embodiment, we want to distinguish question sentences from others. In order to simplify the expression, in the following specification, the question sentence is referred to as “question”, and the rest is referred to as “description”. Assume that each sentence in the bilingual corpus used for training is labeled “question” or “description”.

2. System Overview 2.1 System Architecture FIG. 1 described below shows the overall structure of this system. The data is divided into classes, and further divided into a training set and a development set for each class. Three complete SMT systems are built. One for each class and one for data from both classes. A probability classifier (described in the next section) is also trained from the complete set of training data.

  The machine translation decoder used can linearly interpolate all models from all subsystems according to a vector of interpolation weights given for each source word sequence to be decoded. In order to do this, prior to the search, the decoder must first merge the phrase tables from each subsystem. All of the phrases in all phrase tables are used during decoding. Phrases that appear in the table of one subsystem but not in the tables of other subsystems are also used, but there is no support by the subsystem that did not acquire this phrase during training (zero probability). The search process is performed in the same way as in a typical multistage phrase-based decoder.

  The weight for the general model is set by adjusting this parameter to maximize the BLEU score for the general development set. This weight determines the amount of probability to be assigned to the general model and remains fixed during the decoding of all sentences. The remaining portion of the magnitude of probability is divided among the class specific models dynamically at run time for each sentence. The percentage assigned to each class is simply the class membership probability of the source sentence assigned by the classifier.

3. Question Prediction 3.1 Overview of the Problem Given a certain class of source sentences (questions or narrations in this embodiment), it is desirable to ensure that the generated target sentence is an appropriate class. This does not necessarily mean that if a question is given at the source, the question must be generated at the target. However, at least intuitively, it would be reasonable to assume that the target question should be able to be generated from the source question and the target description from the source description. This is reasonable because the machine translation engine's role is to generate one acceptable translation rather than all possible translations from the source. This assumption leads to the two most appropriate strategies to proceed.

  1. Predict the source sentence class and use it to constrain the decoding process used to generate the target.

  2. Predict the target class.

  In the experiments described below, the second method was chosen because it appeared to be the most accurate, but it appears that both strategies have reasonable advantages.

3.2 Maximum Entropy Classifier In this embodiment, a maximum entropy (ME) classifier is used, and a class to which an input source sentence belongs is determined using a set of lexical feature quantities. That is, the classifier is used to set the mixing weight of the class specific model. Recently, such classifiers have generated powerful models using a large number of lexical features in various natural language processing tasks. See, for example, Ronald Rosenfeld, 1996 (Ronald Rosenfeld. 1996. Maximum entropy approach to adaptive statistical language modeling. Computer speech and language. 10: 187-228) (Ronald Rosenfeld. 1996. A maximum entropy. Computer Speech and Language. 10: 187-228) The ME model is an exponential model of the following form.

ここで、
ｔは予測されるクラス、
ｃはｔの文脈、
γは正規化係数、
Ｋはモデル中の特徴量の数、
α_ｋは特徴量ｆ_ｋの重み、
ｆ_ｋは二次特徴量関数、
ｐ_０はデフォルトモデルであり、
これらはソース文中の、文のクラスを予測するための特徴量である。

here,
t is the predicted class,
c is the context of t,
γ is a normalization factor,
K is the number of features in the model,
α _k is the weight of the feature quantity f _k ,
f _k is a secondary feature function,
p ₀ is the default model,
These are feature quantities for predicting a sentence class in a source sentence.

  Furthermore, in order to distinguish what appears in the sentence from n-grams appearing at the beginning and end of the sentence, we introduced a beginning token (<s>) and a sentence end token in the word sequence. This is because a word indicating that the “question word” or sentence is a question is (for example, wh- <what, where, when> in English, -kah word in Malay- <apakah, dimanakah, kapankah>) ) Based on the observation that it is often found at the beginning of sentences or is often found at the end of sentences (like <ka> in Japanese or <ma> in Chinese).

  The reason why this n-gram extraction is adopted is that an error analysis indicates that n-grams from the inside of the sentence are necessary to handle sentences such as “exclude me please where is ...”. A simple example sentence and a set of feature values generated from the sentence are shown in FIG. 11 and will be described in detail later.

  In order to realize the ME model of the present invention, Le Zhang's ME modeling toolkit was used. (LeZhang. 2004. Maximum entropy modeling toolkit for Python and C ++) (Le Zhang. 2004. Maximum Entropy Modeling Toolkit for Python and C ++, [http://homepages.inf.ed.ac.uk/s0450736/maxent_toolkit. html]). These models were trained by L-BFGS parameter estimation and used a Gaussian prior for smoothing during training. “L-BFGS” is a well-known software package for solving nonlinear optimization problems.

  Take the n best output from the decoder and select the highest translation hypothesis in the list of matching classes according to the source and target classifiers.

4). System Configuration FIG. 1 shows the overall structure of the SMT system 30 of this embodiment. Referring to FIG. 1, the SMT system 30 includes a training module 44 for training a class dependent SMT model, a classifier model used to classify source sentences, and a phrase table used in an SMT decoder. The training set 42 is used as training data. The training module 44 further estimates a weight W1 assigned to the general SMT model. The weight is estimated based on the development set 40. Bilingual corpora are divided into classes, and each class is further subdivided into a training set and a development set.

  The SMT system 30 further includes a statistical machine translation (SMT) device 46 for translating the source language input sentence 48 into a target language translation 50. The SMT device 46 statistically translates based on the model trained by the training module 44 and the weight W1 estimated by the training module 44.

  A training module 44 is a classifier training module for training the classifier model 100 so that, given a set of feature quantities of an input sentence, the probability that the sentence is a question is calculated based on the classifier model 110. 72, three sets of class-dependent SMT models 112, namely, an SMT training module 74 for training models specific to general, question specific and narrative, and a phrase table 114 extracted from the training set 42 of a bilingual corpus. A phrase table generation module 76 for generation and a weight estimation module 70 for estimating the weight W1 assigned to the general set of general SMT models based on the development set 40 are included.

  The SMT device 46 includes a storage unit 90 for storing the classifier model 110, the three sets of the class-dependent SMT model 112, the phrase table 114, and the weight 116 (W1) estimated by the weight estimation module 70.

The SMT device 46 is further based on a classifier 92 that estimates the probability P _Q that the input sentence 48 is a question sentence, a general SMT model during the translation process, a SMT model specific to the question, and a SMT model specific to the description. A normalization module 94 for normalizing the weights W1, W2 and W3 assigned to the probabilities, calculated so that the sum of the weights W1, W2 and W3 is 1, and a source language input sentence 48, a statistical machine And an SMT module 96 for translating into the target language translation 50 using the translation method. The SMT module 96 is a normal SMT module except that the hypothesis probability is calculated by a weighted sum of the probabilities from the three sets of the SMT model 112 instead of the probabilities derived from the general set.

  FIG. 2 is a detailed block diagram illustrating the SMT training module 74 and the three sets of class dependent SMT models 112 of FIG.

  Referring to FIG. 2, the three sets of class-dependent SMT models 112 include a set 160 of general SMT models, a set 162 of SMT models specific to the question, and a set 164 of specific SMT models to the description.

  The general SMT model 160 includes a language model 180, a translation model 182, a length model 184, and a distortion model 186.

  The language model (LM) gives the probability of the appearance of a word under the condition that N-1 other words appear immediately before. The N-gram LM is constructed (trained) from statistics obtained from the target portion of the training set 42 of the bilingual corpus.

  The translation model (TM) gives the probability that a word in the first language is translated into a word in the second language. In this embodiment, TM 182 is obtained from a training set 42 of a bilingual corpus.

  The length model (LeM) gives a penalty for each additional word in the translation (target) relative to the average. The length model 184 is obtained from the target portion of the sentence pair of the bilingual corpus training set 42.

  The distortion model (DM) penalizes the relative distance between two source language phrases associated with two adjacent target language phrases. The DM 186 is statistically obtained from the training set 42 of the bilingual corpus.

  Similarly, the set of SMT models 162 specific to the question includes LM200, TM202, LeM204, and DM206, and the set of SMT models 164 specific to the description includes LM220, TM222, LeM224, and DM226.

  The SMT training module 74 includes a general SMT training module 130 for training a set of general SMT models 160 based on the entire training set 42 and a sentence pair from the training set 42 that includes a question on the target side. A question extraction module 132, a question specific SMT training module 134 for training a SMT model 162 specific to the question based on the sentence pair extracted by the question extraction module 132, and a sentence from the training set 42. A narrative extraction module 136 for extracting a pair including a narrative on the target side, and a narrative for training the SMT model 164 specific to the narrative based on the sentence pair extracted by the narrative extraction module 136 Special And a SMT training module 138.

  FIG. 3 is a block diagram of the phrase table generation module 76 shown in FIG. Referring to FIG. 1, the phrase table generation module 76 includes an automatic alignment module 240 that associates a source sentence and a target sentence of each pair of a bilingual corpus training set 42, and a source sentence associated by the automatic alignment module 240. And a phrase extraction module 242 that identifies the target sentence and extracts the phrase.

  The automatic alignment module 240 associates each word in the source sentence with a corresponding word in the target sentence. The phrase extraction module 242 extracts a specific word sequence in the source sentence, which is associated with consecutive words in the target sentence, as a pair of phrases, and stores them in the general phrase table 244.

  Similarly, the phrase table generation module 76 further includes an automatic alignment module 250 and phrase extraction module 252 for generating the question specific phrase table 254 and an automatic alignment module 260 and phrase extraction for generating the narrative specific phrase table 264. Module 262.

  The phrase table generation module 76 further includes a table merge module 270 for merging the general phrase table 244, the question specific phrase table 254, and the narrative specific phrase table 264. In generating the phrase table 114, phrases that appear in the table of one subsystem but not in the table of another subsystem are also used, but there is no support from subsystems that do not acquire this phrase during training ( Zero probability).

  FIG. 4 is a detailed block diagram of the classifier training module 72 shown in FIG. 1, which receives a predetermined set of input sentence features and the probability that the sentence is a question based on the ME model. For training the ME (maximum entropy) model for the question specific classifier 92.

  Referring to FIG. 4, the classifier training module 72 includes a feature quantity extraction module 290 that extracts a predetermined set of feature quantities from each of the source sentences of the training set 42 of the bilingual corpus, a set of feature quantities, A storage unit 292 that stores the label (question / description) of the source sentence and a maximum entropy modeling module 294 for calculating the probability classification model 110 are included. Maximum entropy modeling module 294 is implemented with a maximum entropy toolkit. Several such toolkits are available on the Internet.

  FIG. 5 is a block diagram of the weight estimation module 70 shown in FIG. Referring to FIG. 5, the weight estimation module 70 uses the bilingual corpus development set 40 and the SMT device 46 so that the average BLEU score calculated for the translation set 310 is the highest, so that the general SMT weight W1. To optimize.

  The weight estimation module 70 includes a BLEU evaluator 320 that evaluates the BLEU score of all translations in the translation set 310. The translation set 310 includes the translation of all source sentences in the development set 40 into the target language by the SMT device 46. The BLEU evaluator 320 uses the target portion of the sentence pairs in the development set 40 as a reference translation.

  The weight estimation module 70 further optimizes the storage unit 322 for storing the BLEU score of the translation evaluated by the BLEU evaluator 320 and the weight 326 (W1) for the general SMT probability by repetition of the translation and the evaluation. A weight optimization module 324. As will be described later, prior to optimization of the weight W1, three sets of a classifier model 110, a class specific SMT model 112, and a phrase table 114 are generated. Thus, optimization of the weight W1 is possible by repeatedly translating all source sentences for each set of weights ranging from 0 to 1 and finding the value that yields the highest BLEU score.

  FIG. 6 is a block diagram of the SMT module 96 shown in FIG. Referring to FIG. 6, the SMT module 96 receives an input sentence 48 and, based on the set of general SMT models 160, outputs a general SMT subsystem 340 that outputs its SMT (SM and TM) probabilities along with LeM and DM penalties. , A weighting module 350 that multiplies each of the probabilities and penalties from the target language by the weight W1 from the normalization module 94 of FIG. 1, and receives the input sentence 48 and, based on the query specific SMT model 162, LeM and DM penalties And the query specific SMT subsystem 342 that outputs the SMT probability and the probability and penalty from the query specific SMT subsystem 342, respectively, and receives the input sentence 48 and includes the LeM and DM penalties based on the description specific SMT model. Weighting mod to output SMT probability 352 and the input sentence 48, and based on the description specific SMT model 164, the description specific SMT subsystem 344 that outputs the SMT probability together with the LeM and DM penalties, and specifies the values of the LM and TM as questions. A weighting module 354 that multiplies with the description.

  The SMT module 96 further receives a sum module 360 that sums the weighted LM, TM, LeM and DM penalties, and the sum of the LM and TM probabilities and the LeM and DM penalties, and uses the phrase table 114. A multi-stage phrase-based decoder 362 that searches for the n-best hypothesis of the translation of the input sentence 48.

  FIG. 7 is a simplified block diagram of the weighting module 352. Referring to FIG. 7, weighting module 352 multiplies LM probability from question specific SMT subsystem 342 by weight W2, and multiplies TM probability from question specific SMT subsystem 342 by weight W2. A multiplier 402, a multiplier 404 that multiplies the LeM penalty from the query specific SMT subsystem 342 by a weight W2, and a multiplier 406 that multiplies the DM penalty from the query specific SMT subsystem 342 by a weight W2.

  Although not shown, the weighting modules 350 and 354 have the same structure as the weighting module 352. However, the weights of the weighting modules 350 and 354 are W1 and W3, respectively. The outputs of the weighting modules 350, 352 and 354 are provided to the summing module 360.

  FIG. 8 is a block diagram of the summing module 360 shown in FIG. Referring to FIG. 6, summing module 360 includes four summing circuits 420, 422, 424 for calculating LM probabilities, TM probabilities, LeM penalties, and DM penalties output from weighting modules 350, 352, and 354, respectively. And 426. The outputs of summing circuits 420, 422, 424 and 426 are provided to the input of decoder 362, which searches for the most probable hypothesis for translation based on these values.

FIG. 9 shows that the weights W2 and W3 are normalized based on the probability P _Q estimated by the classifier 92 so that the sum of the weights W1, W2, and W3, which are elements of the weight vector representing class membership, becomes 1. It is a block diagram of the normalization module 94 for The weight W1 remains fixed once optimized by the weight estimation module 70. Therefore, normalization module 94 is such that the sum of the W2 and W3 is 1-W1, normalizing and _{P Q} and _{1-P Q} for W2 and W3.

Specifically, the normalization module 94 is coupled to the storage unit 440 for storing the numerical constant “1” so that one input receives the probability P _Q from the classifier 92 and the other input is the storage device 440. And a subtractor 442 that outputs the difference between the constant 1 and the probability P _Q , that is, 1−P _Q , one input coupled to receive the weight W 1, and the other input coupled to the storage device 440. A subtractor 444 that outputs the difference between the constant 1 and the weight W1, one input coupled to receive the output of the subtractor 444 and the other input coupled to receive the probability P _Q from the classifier 92. Multiplier 446 and a multiplier 448 having one input coupled to receive the output of subtractor 444 and the other input coupled to receive the output of subtractor 442.

Each output of the subtractor 442 and 444, is equal to the _{1-P Q} and 1-W1. Accordingly, the outputs W2 and W3 of the multipliers 446 and 448 are equal to P _Q * (1−W1) and (1−P _Q ) * (1−W1), respectively. The sum of W1, W2 and W3, ie W1 + P _Q * (1−W1) + (1−P _Q ) * (1−W1) is equal to 1.

FIG. 10 is a block diagram of the classifier 92 shown in FIG. Referring to FIG. 10, the classifier 92 includes a feature quantity extraction module 460 for extracting from the input sentence 48 the same feature quantity set extracted by the feature quantity extraction module 290 shown in FIG. model 110 based on the characteristic quantity of the set of input text 48 that is extracted by (see FIG. 1) and the feature extraction module 460 includes a probability calculation module 462 for calculating the probability P _Q of the input sentence 48, the .

  FIG. 11 shows a set of n-grams (n ≦ 3) extracted from the sentence “<s> where is the station </ s>” used as a predicate in the ME model to predict the class of the target sentence. This set consists of 4 unigrams (<s> where, is, the, station </ s>), 3 bigrams (<s> where is, is the, the station </ s>), and 2 Of the trigram (<s> where is the, is the station </ s>). In order to simplify the description of the feature quantity of n-grams, n is 3 in FIG. However, the number of n is not limited to three. As will be described later, the inventors use a 5-gram feature (n = 5) in the experiment.

5. Operation <Overall procedure>
The SMT system operates as follows. The SMT system 30 generally includes two operational phases. Training stage and translation stage.

  Referring to FIG. 12, the training stage includes four sub-stages. Training of the class dependent SMT model 112 (step 500), training of the classifier model 110 (step 502), generation of the phrase table 114 (steps 504 and 506), and the weight W1 for the general model of the development set 40 And optimization (step 508). When steps 500 to 508 are complete, the SMT system 30 is ready to translate any input sentence.

[SMT model training (step 500)]
With reference to FIG. 2, the general SMT training module 130 trains the general SMT model 160 based on all data in the training set 42. Training of the SMT model is performed in the usual way.

  The question extraction module 132 extracts from the training set 42 pairs of sentences each including a question sentence on the target side. The question specific SMT training module 134 trains the question specific SMT module 162 based on the sentence pairs extracted by the question extraction module 132. The training method is the same as that of the general SMT training module 130.

  The narrative extraction module 136 extracts from the training set 42 a pair of sentences each including a narrative sentence on the target side. The narrative specific SMT training module 138 trains the narrative specific SMT module 164 based on the sentence pairs extracted by the narrative extraction module 136. The training method is the same as that of the SMT training module 130 and the question specifying SMT training module 134.

[Training of classifier model 110 (step 502)]
Referring to FIG. 4, the feature quantity extraction module 290 extracts the same set of feature quantities as extracted by the feature quantity extraction module 460 shown in FIG. 10 from each of the source sentences of the pair of sentences in the training set 42. . The storage unit 292 stores the extracted feature value pairs together with the sentence labels (questions / descriptions) of the sentences on the target side. Thereafter, the maximum entropy modeling module 294 calculates the parameters of the classification model 110 according to the equation (1) based on the set of feature values and the sentence label stored in the storage unit 292.

[Phrase Table Generation (Steps 504 and 506)]
Referring to FIG. 3, the automatic alignment module 240 associates a source sentence word and a target sentence word for each sentence pair of the training set 42. The phrase extraction module 242 extracts a phrase pair from the associated sentence pair. Here, the phrase extraction module 242 finds a sequence of consecutive words in the source sentence associated with the consecutive words in the target sentence, and extracts a pair of these word sequences as a phrase translation pair. The extracted phrase pairs are stored in the general phrase table 244.

  The automatic alignment module 250 associates the words of the source sentence with the words of the target sentence in each sentence pair labeled “Question” in the training set 42. Similarly to the general phrase table 244, the phrase extraction module 252 extracts phrase pairs from the associated sentence pairs. The extracted phrase pairs are stored in the question specific phrase table 254.

  The automatic alignment module 250 associates the source sentence word with the target sentence word in each pair of sentences labeled “description” in the training set 42. Similar to the phrase extraction module 242 and the general phrase table 244, the phrase extraction module 262 extracts phrase pairs from the associated sentence pairs. The extracted phrase pairs are stored in the description specific phrase table 264.

  The table merge module 270 merges the general phrase table 244, the question specific phrase table 254, and the narrative specific phrase table 264. Here, phrase pairs appearing in one or two of the tables 244, 254 and 264 are stored in the phrase table 114. However, subsystems that did not acquire this phrase during training have no support (zero probability).

[Optimization of weight W1 (step 508)]
The development set 40 is used for optimizing the weight W1. Referring to FIG. 5, each of the source sentences in the development set 40 is translated by the SMT device 46 to form a translation set 310. A BLEU evaluator 320 evaluates each BLEU score of the translation. The target sentence in the development set 40 is used as a reference translation in this evaluation. The average of the BLEU score is calculated and stored.

  In the next cycle, the value of the weight W1 is changed slightly, and the same BLEU evaluation is performed. Thus, by minimum error training (Franz J. Och, 2003. Minimum error rate training for statistical machine translation, Proceedings ACL. ), The weight W1 of the general model is optimized.

  Once optimized, weight W1 remains fixed during sentence decoding (translation).

[Translation by SMT module 96]
When an input sentence 48 without a label (question / description) is given to the classifier 92 (see FIGS. 1 and 10), the feature amount extraction module 460 extracts a feature amount pair from the input sentence 48, and the feature amount is extracted. Are provided to the probability calculation module 462. The probability calculation module 462 calculates the probability that the input sentence 48 is a question by applying the set of feature quantities to the classifier model 110. The calculated probability P _Q is provided to the inputs of the subtractor 442 and the multiplier 446 of the normalization module 94. Based on the probability P _Q given from the classifier 92, the normalization module 94 normalizes the weights W2 and W3 such that the sum of the weights W1, W2 and W3 is 1, and the weights W1, W2 and W3 are SMT. To module 96.

  Referring to FIG. 6, the general SMT subsystem 340, the question specifying SMT subsystem 342, and the description specifying SMT subsystem 344 are given a general SMT model 160, a question specifying SMT model 162, and a description specifying given a set of features. Based on the SMT model 164, hypothesis probabilities are calculated independently. LM and TM probabilities and LeM and DM penalties are provided from the general SMT subsystem 340, the query specific SMT subsystem 342 and the narrative specific SMT subsystem 344 to the weighting modules 350, 352 and 354, respectively, and weights W1, W2 and Each is weighted by W3.

  The weighted LM and TM probabilities and the weighted LeM and DM penalties are provided to the sum module 360 (see FIG. 8), where the LM probabilities from the weight modules 350, 352 and 354 are added. . Similarly, TM probabilities from weighting modules 350, 352 and 354 are added. The LeM probability and DM penalty are added in the same way. The LM probability, TM probability, LeM penalty and DM penalty obtained in this way are given to the decoder 362.

  Based on these values, the decoder searches for the most likely hypothesis for the translation of the input sentence 48 and outputs the n best hypothesis.

6). Experiment 6.1 Experimental data To evaluate the proposed technology, an experiment was conducted on a travel conversation corpus. The experimental corpus is a travel component of the BTEC corpus (Kikui et al., 2003. Generating a corpus for speech-to-speech translation. Eurospeech Proceedings, pages 381-384), (Kikui, et al., 2003. Creating Corpora for Speech-to-Speech Translation. In Proceedings of EUROSPEECH, pages 381-384) Targeted English and each of the other languages as the source language. Training, development, and evaluation corpus statistics are as shown in Table 1. In the evaluation corpus, there are 16 reference translations per sentence.
(Table 1)

データはクラスに分けられ（質問及び叙述）、さらに各クラスについてトレーニングセットと開発セットとに細分された。１０００個の文が開発データとして取除けられ、残りがトレーニングに用いられた。

The data was divided into classes (questions and descriptions) and further subdivided into a training set and a development set for each class. 1000 sentences were removed as development data, and the rest was used for training.

  Experiments were conducted on a variety of different languages. These are represented by the following keys: Arabic (ar), Danish (da), German (de), English (en), Spanish (es), French (fr), Indonesian (Malay) (id ), Italian (it), Japanese (ja), Korean (ko), Malaysian (Malay) (ms), Dutch (nl), Portuguese (pt), Russian (ru), Thai (Th), Vietnamese (vi), and Chinese (zh).

[decoder]
The decoder used in the experiment, CleopATRa, is PHARAOH (Pharaoh) (Philip Cohen, 2004. Pharaoh: a beam search decoder for phrase-based statistical machine translation models. Machine translation: from real users to research: 6th AMTA Conference, Washington DC, Springer Ferrack, pages 115-124) (Philipp Koehn. 2004. Pharaoh: a beam search decoder for phrase-based statistical machine translation models. Machine translation: from real users to research: 6th conference of AMTA, Washington, DC, Springer Verlag, pp. 115-124. and MOSES (Philip Cohen et al., 2007. Moses: open source toolkit for statistical machine translation, ACL 2007: demos and posters Session proceedings, Prague, Czech Republic Pp.177-180 (Philipp Koehn et al., 2007. Moses: open source toolkit for statistical machine translation, ACL 2007: proceedings of demo and poster sessions, Prague, Czech Republic, pp. 177-180.) A phrase-based statistical decoder within the applicant's organization that operates on the same principle. In these experiments, the decoder was configured to produce almost the same output as MOSES. The decoder was modified to handle multiple sets of models, accept weighted inputs, and incorporate dynamic interpolation during decoding.

[Practical problems]
Of most concern for the proposed approach is the excessive demand for resources that can arise when dealing with a large number of models. However, one important feature of the decoder used in this experiment is the ability to place the model on disk and load only the part of the model necessary to decode the sentence at hand. This reduces memory overhead when loading a large number of models without appreciably degrading the decoding time. In addition, the interpolability can be calculated in advance for most of each sentence model before the search is started, thereby reducing both the search memory and the processing time.

[Decoding conditions]
For the adjustment of the decoder parameters, the minimum error training for the BLEU score was performed using the respective development corpus. SRI Language Modeling Toolkit (Andreas Stolk 1999. SRILM-Extensible Language Model Toolkit) (Andreas Stolcke. 1999. SRILM-An Extensible Language Model Toolkit. Http://www.speech.sri.com/projects/srilm/ And a 5 gram language model constructed using Witton-Bell smoothing. The model includes a length model and also includes a simple distance-based distortion model used in the PHARAOH decoder.

[Interpolation weight adjustment]
The interpolation weights were adjusted by maximizing the development set BLEU score by a set of weights incrementing by 0.1 in the range of 0 to 1. FIG. 13 shows the behavior of the two models of the present invention with respect to the weight parameter.

  Referring to FIG. 13, the BLEU score for translation from Chinese (zh) to English indicated by a broken line 522 did not improve even when the weight W1 was increased from zero. On the other hand, in the case of translation from Indonesian (Malay) (id) to English indicated by a solid line 520, the BLEU score was maximized when W1 was about 2. This indicates the dependency of this system on the combination of source and target languages.

[Evaluation scheme]
In order to see the benefits of the proposed approach in a balanced manner, six different evaluation techniques were used in the experiment to evaluate this system. That is, BLEU (Kisho Papineni et al., 2001. Bleu: automatic evaluation method of machine translation. IBM research report, RC22176, September 17) (Kishore Papineni et al., 2001. Bleu: a method for automatic evaluation of machine translation IBM Research Report, RC22176, September 17.), NIST (George Dodington, 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. Proc. Of Human Language Technology Conference, San Diego, California, 138-145. Page) (George Doddington. 2002 Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. Proceedings of Human Language Technology Conference, San Diego, California, pp. 138-145.), WER (Word Error Rate: word error) Rate), PER (Position independent WER: rank) Independent WER), GTM (General Text Matcher), and METEOR (Sataniev Banerigi and Aron Rabi, 2005. Automatic metrics for MT evaluation with improved correlation with human judgment, ACL-2005: Workshop on intrinsic and external assessment scales for machine translation and / or summaries, pages 65-72) (Satanjeev Banerjee and Alon Lavie. 2005. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments ACL-2005: Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and / or Summarization, pp. 65-72.

6.2 Classification accuracy Table 2 shows the performance of the classifier (by 10-fold cross-validation of the training set). The numbers of classification accuracy to predict the punctuation of source (same language) and target (English) are shown. Of course, on all systems, its own punctuation was better predicted. A bad score in the table will probably reflect a linguistic characteristic (perhaps the source sentence question is often expressed as a statement in the target) or a characteristic of the corpus itself. For all languages, the accuracy of the classifier seems to be satisfactory, especially considering that the corpus itself may be inconsistent (and so is the test data in this experiment).

６．３翻訳の品質
ＳＭＴシステムの性能を表３に示す。

6.3 Translation quality Table 3 shows the performance of the SMT system.

この表から、評価された実験条件のほとんどについて、全データでトレーニングされたＳＭＴシステムからなるベースラインシステムにくらべ、このシステムの性能が勝っていることが明らかである。性能が劣化している数値部分では、１つを除く全てで、結果は統計的には有意なものではなく、全ての事例で、他のＭＴ評価メトリックスは改善を示した。いくつかの言語対では驚くべき改善が見られ、特に、この技術を用いると、マレー語ｉｄとｍｓはいずれもＢＬＥＵが３．５ポイントも改善された。

From this table it is clear that for most of the experimental conditions evaluated, the performance of this system is superior to a baseline system consisting of an SMT system trained with all data. In the numerical part where performance was degraded, the results were not statistically significant in all but one, and in all cases the other MT evaluation metrics showed improvement. Some language pairs have seen remarkable improvements, especially with this technique, both Malay id and ms have improved BLEU by 3.5 points.

  Interestingly, the Malay relative, Dutch, has also improved substantially. This provides linguistic explanations for gain. Malay has a very concise and regular question structure, where the question word appears at the beginning of the question sentence (similar to the target language) and performs other functions in that language (unlike English “do”, for example) There is nothing. Perhaps because of the simplicity of this representation, the class specific model of the present invention was able to model the data well despite the data being reduced due to the data partitioning.

  Another factor seemed to be classifier performance, which was high in all languages (about 98%). Unfortunately, it is difficult to know the reason behind the diversity of table scores. One of the major factors may be the difference in corpus quality and the relationship between the source corpus and the target corpus. Some corpora are direct translations of each other, others are multiple translations from different languages. This is the reason why Chinese was one of these languages, and although it was very successful in Japanese and Thai, which are closely related to Chinese, this language did not improve from the baseline. I may be able to do it.

[Comparison with the previous method]
An experiment was conducted to compare the proposed method with an implementation using the hardware weight of this system. The aim was to try to get as close as possible to the system proposed in the prior art Non-Patent Document 1 within this framework. Instead of weighting the class specific model with classification probability, weights of 1 and 0 were used. In order to achieve this, the probability from the classifier was binarized so that a weight of 1 was given if the probability was> 0.5, and a weight of 0 was used otherwise. The performance of this system is shown in the column labeled “Hard” in Table 4. Under all conditions except one, the approach proposed in the invention outperformed or was better than this system.

表４の、「分類器なし」のラベルの欄は、発明のシステムの分類器の有効性を示している。これらの結果から、質問モデルと叙述モデルとの間の補間に等しい重み（０．５）を用いる効果が示された。このシステムは、分類器を用いたシステムほどではないが、相当の性能を示した。

The column labeled “No Classifier” in Table 4 shows the effectiveness of the classifier of the inventive system. These results showed the effect of using equal weight (0.5) for the interpolation between the query model and the narrative model. This system did not perform as well as a system using a classifier, but showed considerable performance.

7). CONCLUSION In the above-described embodiment, the two models from the SMT engine specific to the question and the SMT engine specific to the description were combined into a single decoding process. However, the present invention is not limited to two classes of systems. As is apparent from Equation 1, the present invention is applicable to systems that include three or more classes.

  This technique enables topic-dependent decoding with flexible weighting based on the probability between component models. The experiment showed the effectiveness of the embodiment of the present invention for conversation data by constructing a class specific model in the class of question sentences and narrative sentences. Extensive evaluation of the technology using numerous language pairs and MT evaluation metrics demonstrates the effectiveness of the present invention. In most cases, a significant improvement over systems without model interpolation can be shown, and this approach is superior for some language pairs. The most improved of all language pairs was Malaysian (Malay) and English, increasing the BLEU by 4.7 points (0.463 to 0.510) over the baseline system.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is an overall block diagram of an SMT system 30 according to an embodiment of the present invention. FIG. 2 is a detailed block diagram of the three sets of class dependent SMT models 112 and the SMT training module 74 shown in FIG. 4 is a detailed block diagram of a phrase table generation module 76. FIG. 4 is a detailed block diagram of classifier training module 72. FIG. FIG. 6 is a block diagram of a weight estimation module 70. 3 is a detailed block diagram of an SMT module 96. FIG. FIG. 6 is a simplified block diagram of a weighting module 352. FIG. 3 is a simplified block diagram of a summing module 360. 4 is a block diagram of a normalization module 94. FIG. 6 is a simplified block diagram of classifier 92. FIG. It is a figure which shows the example of the group of n-gram feature-value extracted from the sentence of "<s> where is the station </ s>". 3 is a flowchart showing an operation process of the SMT system 30. It is a figure which shows the behavior with respect to those weight parameters of two models used in the experiment, namely, Chinese (zh) and Indonesian (id).

Explanation of symbols

30 SMT system 40 Development set 42 Training set 44 Training module 46 SMT device 48 Input sentence 50 Translation 70 Weight estimation module 72 Classifier training module 74 SMT training module 76 Phrase table generation module 92 Classifier 96 SMT module 110 Classifier model 112 Class Specific SMT Model 114 Phrase Table 130 SMT Training Module 134 Question Specific SMT Training Module 138 Descriptive Specific SMT Training Module 160 General SMT Model 162 Question Specific SMT Model 164 Descriptive Specific SMT Model 290 and 460 Feature Extraction Module 294 Maximum Entropy Modeling Module 324 Weight Optimization module 340 General SMT sub The stem 342 questions specific SMT subsystem 344 narrative certain SMT subsystem 362 decoder

Claims (6)

Includes means for determining a vector of probabilities representing the class membership of the source sentence, the elements of the vector represents the probability of belonging respectively to the plurality of classes the source sentence predetermined further
Wherein provided for each plurality of classes, look including a plurality of sub Budekoda, the plurality of sub-decoders are statistically trained by collection of training data for each class, of the plurality of sub-decoder each of the each of the words and certain phrases of the source sentence, and outputs the probability of the translation word and translation word sequences in the target language, respectively,
According to the probability of the hypothesis obtained by combining the target language translation word and translation word sequences, means for outputting a high anchor theory of most likelihood in the target language of the source text as a translation for the source text further comprising a, the probability of the hypothesis of the target language, the probability output by the plurality of sub-decoders for each of the target language translation word and translation word sequence which forms the hypothesis before Kibe vector A statistical machine translation apparatus that is calculated using a value obtained by adding elements as weights . The plurality of classes includes a general class and a plurality of specific classes,
The statistical machine translation device according to claim 1, wherein the plurality of specific classes are obtained by dividing the general class. The statistical machine translation device according to claim 1 or 2, wherein one element of the vector corresponding to the general class is a constant in a range of 0 to 1. The statistical machine translation apparatus according to claim 1, further comprising normalization means for normalizing elements of the vector so that a sum of the elements becomes 1. 5. Means for determining the pre Kibe vector is statistically trained on maximum entropy models, assigning a membership probability to each of the classes, statistical according to any one of claims 1 to 4 Machine translation device. Each of said plurality of sub Budekoda the class specific language model, class specific translation model, calculates the probability according to any combination of classes specified length model, or class specific distortion model or these models, claims 1 Item 6. The statistical machine translation device according to any one of Items 5.

Images