JP6040946B2 - Word order rearrangement device, translation device, method, and program - Google Patents

Description

  The present invention relates to a word order rearrangement device, a translation device, a method, and a program, and more particularly, to a word order rearrangement device, a translation device, a method, and a program for rearranging the word order of an input sentence.

  The process of machine translation from language A to language B is roughly divided into two: translation from language A phrases to language B phrases, and appropriate reordering of translated language B phrases in language B. The In the statistical translation technology widely used in the field, the translation of words and the rearrangement of words are statistically modeled from the correspondence between the words of language A and the words of language B estimated from a large number of parallel translations. The language B input sentence is searched for a translation sentence of the language B constituted by translation of a probable phrase and rearrangement of the phrase based on the statistical model.

  In general, it is extremely difficult to comprehensively search for all translation candidates, so the number of translation candidates for each word is limited, and the distance of word sorting is limited within a certain range. By doing so, machine translation processing with a practical calculation amount is realized.

  However, depending on the combination of language A and language B to be translated, the corresponding words may appear in a significantly different order, and the reordering is large enough to accurately translate between such languages. Since translation processing in consideration of distance is required, there is a problem that an increase in calculation amount is unavoidable.

  As a technique for coping with the above problem, a technique called “pre-ordering” is known in which the words of language A are rearranged so as to approach the order of the corresponding words of language B before translation processing is performed. (Patent Document 1, Non-Patent Document 1). The method of Non-Patent Document 1 is intended for translation from German to English, and the method of Patent Document 1 is intended for translation from English to Japanese. The language of the input sentence (language A) is the language after translation (language B). It uses a rule that rearranges the words so that they match the order of the corresponding phrases.

  Also, in Japanese-to-English translation, there is a known technology that uses syntactic analysis to estimate the dependency structure of Japanese clauses, and changes the order of English clauses to bring them closer to the English word order (non-patented). Document 2, Non-Patent Document 3, Non-Patent Document 4).

  In the method of Non-Patent Document 2, in addition to the dependency structure, the subject and object are identified using a technique for estimating the subject and object for a predicate called predicate term structure analysis, and the English subject-verb- The rules for moving the subject and object in order of the object are used.

  The method of Non-Patent Document 3 estimates the subject and object using a dependency structure and particles, and uses the same rearrangement rule as the method of Non-Patent Document 2. These methods can be expected to arrange the order of clauses in the order close to English, but the word order in the clauses does not change, so the phrase `` I arrived in Tokyo '' will only be in the order of `` I arrived in Tokyo ''. , The position of the particle “ni” and the preposition “at” remains different from the corresponding English “arrived at Tokyo”.

  The method of Non-Patent Document 4 improves the method of Non-Patent Document 2 and moves the function words such as Japanese particles and auxiliary verbs to the beginning of the phrase, so that the positions of the English prepositions are substantially matched. Realization of replacement. A technique is also known in which Japanese predicates are reversed to bring them closer to the English word order without using parsing (Non-Patent Document 5).

JP 2011-175500 A

Michael Collins, Philipp Koehn, Ivona Kucerova, "Clause Restructuring for Statistical Machine Translation", In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp. 531-540, 2005 Mamoru Komachi, Yuji Matsumoto, Masaaki Nagata, "Phrase Reordering for Statistical Machine Translation Based on Predicate-Argument Structure", In Proceedings of International Work-shop on Spoken Language Translation (IWSLT 2006), 2006 Katsuhito Sudoh, Kevin Duh, Hajime Tsukada, Masaaki Nagata, Xianchao Wu, Takuya Matsuzaki and Jun'ichi Tsujii, "NTT-UT Statistical Machine Translation in NTCIR-9 PatentMT", In Proceedings of NTCIR-9, 2011. Sho Hoshino, Yusuke Miyao, Katsuhito Sudoh, Masaaki Nagata, "Two-Stage Pre-ordering for Japanese-to-English Statistical Machine Translation", In Proceedings of IJCNLP, 2013 Jason Katz-Brown and Michael Collins, "Syntactic reordering in preprocessing for Japanese → English translation: MIT system description for NTCIR-7 patent translation task", In Proceedings of the NTCIR-7 Workshop Meeting, 2008.

  However, the methods of Non-Patent Document 1 and Patent Document 1 can be performed fairly accurately by using syntax analysis and appropriate rules on the language A side, but are necessary if the languages A and B are different. There is a problem that it is necessary to define a new rule because different rules are different.

  In addition, the methods described in Non-Patent Document 2, Non-Patent Document 3, and Non-Patent Document 4 enable grammatically valid rearrangement, while processing such as Japanese dependency analysis and predicate term structure analysis. There is a problem of being strongly affected by accuracy. In addition, in order to perform such an analysis, it is necessary to design appropriate rules and to develop a learning corpus, and if the processing target field is different from that at the time of design, it is affected by differences in vocabulary and style. Therefore, there is a problem that the analysis accuracy may be lowered.

  The method described in Non-Patent Document 5 has no problem of analysis accuracy, but in order to simply reverse the word order of the predicate, it arranges in reverse order up to words with order such as parallel phrases and compound words. In other words, the unit order of verbs and objects is almost similar to English, but the internal word order is reversed from the desired English word order.

  The present invention has been made to solve the above problems, and an object thereof is to provide a word order rearrangement device, a translation device, a method, and a program that can appropriately rearrange the word order of an input sentence. .

  In order to achieve the above object, a word order rearranging device according to a first aspect of the present invention identifies, for an input sentence or phrase, a phrase identification unit that identifies each phrase including a functional word, and identifies a main part of the input sentence or phrase Reordering the main sentence identifying part and the main part of the input sentence or phrase identified by the main part identifying part so as to reverse the order of each clause included in the main part, Among them, the part different from the main part is rearranged so as to reverse the order of each phrase, each phrase rearranged for the main part and each phrase rearranged for a part different from the main part, A rearrangement unit that rearranges the word order in the clause so that the function words included in the clause are rearranged at the head of the clause.

  A word order rearrangement method according to a second invention is a word order rearrangement method in a word order rearrangement device including a phrase identification unit, a main part identification unit, and a rearrangement unit, wherein the phrase identification unit is an input For each sentence or phrase, each clause including a function word is identified, the main part identification unit identifies the main part of the input sentence or phrase, and the rearrangement unit is identified by the main part identification unit The main part of the input sentence or phrase is rearranged so that the order of each clause included in the main part is reversed, and the order of each phrase is reversed in a part different from the main part in the input sentence or phrase. The function words included in the clauses are rearranged at the head of the clauses for each clause rearranged for the main part and for each clause rearranged for a part different from the main part. , Rearrange the word order in the clause That.

  According to the first and second inventions, the phrase identifying unit identifies each phrase including a function word for the input sentence or phrase, and the main part identifying part identifies each main part of the input sentence or phrase. , By the reordering unit, the main part of the identified input sentence or phrase is rearranged so as to reverse the order of each clause, the part different from the main part is rearranged so as to reverse the order of each clause, For each sorted clause, the function words contained in the clause are sorted at the beginning of the clause.

  In this way, the order of each clause in the main part is rearranged to be reversed, the order of each phrase in the part different from the main part is rearranged, and the function words of each clause are arranged at the beginning of the phrase. By changing the order, the word order of the input sentence or phrase can be rearranged appropriately.

  In the word order rearrangement device according to the first aspect of the present invention, the word order rearrangement device further includes a parallel identification unit that identifies a combination of clauses in parallel relation from each clause of the input sentence or phrase, and the rearrangement unit includes the parallel A part different from the main part is rearranged so as to reverse the order of the clauses included in the main part so as to maintain the order of the clauses for the combination of clauses in parallel relation identified by the identification part. Rearrange the order of each clause with respect to each other, and for the combination of clauses in the parallel relationship, rearrange the function word included in the most backward clause at the beginning of the most forward clause, For each clause that is different from the combination of clauses in parallel relation, the word order in the clause may be rearranged so that the functional words included in the clause are rearranged at the beginning of the clause. .

  A translation device according to a third invention is a translation device for translating an input sentence or phrase described in a first specific language into a sentence or phrase described in a second specific language different from the first specific language. The result of rearranging the input sentence or phrase by the word order rearranging device of the first invention is described in the second specific language based on a plurality of types of translation models and a weight for each of the plurality of types of translation models. And a translation unit that translates the text into phrases or phrases.

  A translation method according to a fourth aspect of the invention translates an input sentence or phrase described in a first specific language, including a translation unit, into a sentence or phrase described in a second specific language different from the first specific language. A translation method in a translation device, wherein the translation unit rearranges the input sentence or phrase by the word order rearrangement method of the second invention, and includes a plurality of types of translation models and a plurality of types of translation models. Based on the weight for each, the sentence is translated into a sentence or phrase described in the second specific language.

  According to the third and fourth inventions, the translation unit rearranges the input sentences or phrases described in the first specific language by the word order rearrangement device, and the plural types of translation models and the plural types of translation models. Are translated into sentences or phrases written in the second specific language.

  In this way, the order of each clause in the main part is rearranged to be reversed, the order of each phrase in the part different from the main part is rearranged, and the function words of each clause are arranged at the beginning of the phrase. By translating and translating, translation can be performed with high accuracy.

  Moreover, the program of this invention is a program for functioning a computer as each part which comprises said word order rearrangement apparatus.

  The program of the present invention is a program for causing a computer to function as each part constituting the above translation apparatus.

  As described above, according to the word order rearrangement apparatus, method, and program of the present invention, rearrangement is performed so as to reverse the order of each clause in the main part, and the order of each phrase in a part different from the main part is changed. By rearranging so as to be reversed and rearranging the functional words of each clause at the beginning of the clause, the word order of the input sentence or phrase can be appropriately rearranged.

  Further, according to the translation apparatus, method, and program of the present invention, rearrangement is performed so that the order of each clause in the main part is reversed, and rearrangement is performed so as to reverse the order of each phrase in a part different from the main part. By rearranging the function words of each clause at the head of the clause and translating the rearranged sentence or phrase, the translation can be performed with high accuracy.

It is a block diagram which shows the functional structure of the translation apparatus which concerns on embodiment of this invention. It is a block diagram which shows the functional structure of the translation model learning apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the translation model learning process routine in the translation model learning apparatus which concerns on embodiment of this invention. It is a flowchart figure which shows the translation processing routine in the translation apparatus concerning embodiment of this invention. It is a figure which shows the example of the analysis result of KNP. It is a figure which shows the example of interpretation of the analysis result of KNP. It is a figure which shows the example of a sentence division | segmentation. It is a figure which shows the example of identification of a principal part. It is a figure which shows the example which rearranged the clause using the rearrangement rule of a clause. It is a figure which shows the example which rearranged the function word at the head of the clause.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Principle of the present invention>
In the embodiment of the present invention, a sentence written in Japanese is divided into a main part and a predicate in the same manner as in Non-Patent Document 5 in order to perform pre-ordering for translation from Japanese to English. Thereafter, rearrangement in two steps between clauses and within clauses is performed in the same manner as in Non-Patent Document 4 above. In particular, in the present embodiment, excessive order reversal is suppressed by using parallel phrase information so that the order of parallel phrases is not mistakenly reversed in rearrangement between phrases. Japanese is an example of the first specific language, and English is an example of the second specific language.

  In the method of Non-Patent Document 4, the rearrangement of clauses is limited within the range of the dependency structure. However, in this embodiment, only the information on the clause is used, and the dependency structure is rearranged. Do not use for.

  Further, in the method of Non-Patent Document 5 described above, word order reversal (for example, compound words) is performed by performing two-step rearrangement between clauses and within clauses, in which the word order is simply reversed. , “National Institute of Informatics” → “National Institute of Informatics Information”).

  Further, in this embodiment, when there is a parallel structure, the parallel order is not reversed by mistake, thereby reversing the order of the parallel phrases (for example, “device 1 and device 2” → “is device 2 and device 1”). ).

<Configuration of Translation Apparatus According to Embodiment of the Present Invention>
Next, the configuration of the translation apparatus according to the embodiment of the present invention will be described. As shown in FIG. 1, a translation apparatus 100 according to an embodiment of the present invention is a computer that includes a CPU, a RAM, and a ROM that stores a program for executing a translation processing routine described later and various data. Can be configured. The translation apparatus 100 functionally includes an input unit 10, a calculation unit 20, and an output unit 90 as shown in FIG.

  The input unit 10 receives a Japanese input sentence from an input device such as a keyboard. The input sentence is pre-processed by removing tags such as HTML and XML, normalizing the notation, and the like, and is input in a state of being converted into an appropriate input format at the time of input. Note that the input unit 10 may accept input from the outside via a network or the like.

  The calculation unit 20 includes a language analysis unit 30, a rearrangement unit 50, a translation unit 52, and a model storage unit 54.

  The language analysis unit 30 includes a morpheme analysis unit 32, a phrase identification unit 34, a parallel identification unit 36, a sentence division unit 38, and a main part identification unit 40. The language analysis unit 30 performs morphological analysis, phrase identification, phrase combination identification, sentence division, and main part identification for an input sentence described in Japanese received by the input unit 10. Do.

The morpheme analysis unit 32 uses a known morpheme analyzer (such as JUMAN, MeCab) for the input sentence written in Japanese accepted by the input unit 10 and performs morpheme analysis (word break and part of speech identification). Do. In this embodiment, JUMAN is used.

  The phrase identifying unit 34 identifies a phrase for a sentence described in Japanese that has been morphologically analyzed by the morphological analyzer 32. Specifically, in the present embodiment, a known Japanese analyzer (KNP) is used to parse a sentence written in Japanese that has been morphologically analyzed, and the result of the parse analysis. Using only the clause information, for each clause, each word as an element in the clause is used only as a "content word" that has meaning in the word alone and in the form attached to the content word. A classification of “function words” representing the syntactic or semantic role of the content words to be given. There may be a phrase in which only the content word is identified or a phrase in which only the function word is identified. The classification of “content words” and “function words” may be performed by identifying function words using rules in which the function words are fixed to specific parts of speech such as “particles and auxiliary verbs”, or learning from a corpus. You may use the classifier.

  The phrase may be identified using a known Japanese analyzer (CaboCha, etc.), or the phrase may be identified using a simple rule such as the sentence “divide by a functional word such as a particle”. Identification may be performed.

  The parallel identification unit 36 identifies a combination of clauses that have a parallel relationship from each clause of the input sentence described in Japanese identified by the clause identification unit 34. In the present embodiment, the result of syntax analysis by a known Japanese analyzer (KNP or the like) acquired by the phrase identification unit 34 is used. Note that the combination of clauses that are in parallel relation may be identified using rules based on conjunctions and conjunctions, or using a parallel structure analysis program.

  The sentence division unit 38 virtually divides the input sentence described in Japanese at punctuation marks to obtain a plurality of partial sentences. Specifically, the result of syntax analysis by a known Japanese analyzer (KNP or the like) acquired by the phrase identification unit 34 is used. This is because, in Japanese, the structure of a sentence may be largely divided by punctuation marks, and there is often no need to perform long-range sorting across punctuation marks in English translation. It is based on the fact that it may be sufficient to limit the range between the beginning, the end of a sentence, and punctuation marks. For this reason, in the present embodiment, the sentence is virtually divided and divided into a plurality of partial sentences, so that rearrangement beyond the division point (partial sentence range) may be restricted. it can.

  The main part identifying unit 40 identifies a main part included in an input sentence written in Japanese. Specifically, by using the result of syntactic analysis by a known Japanese analyzer (KNP or the like) acquired by the phrase identifying unit 34, a phrase that conforms to the rule of a phrase including the case particles “ha” and “ga” is selected. Identify as the main part. For identification of the main part, a rule such as “use case particles” may be used, or a classifier learned from a corpus may be used. In addition, the main part includes the (main part) phrase including “ha” and “ga” and all the phrases between the preceding division points. When both “ha” and “ga” are included, the main part corresponding to “ha” and the main part corresponding to “ga” may be treated as different main parts. , It may be treated as one main part as a whole.

  The reordering unit 50 identifies each of the clauses included in the input sentence described in Japanese obtained by the language analyzing unit 30, the identification result of the combination of clauses in parallel relation, the identification result of the main part, Using the result of segmentation into partial sentences, for sentences written in Japanese, in the partial sentence unit, rearrange the phrases and the words in the phrase so that they are similar to the word order of the sentences written in English, Combine sorted sub-sentences. Specifically, the following processing is performed.

  First, in the partial sentence unit, the parts identified as the main part by the main part identifying unit 40 are rearranged so that the order of the phrases included in the part is reversed. Note that the combination of clauses that are in parallel relation with the parallel identification unit 36 and the combination of clauses that are identified are rearranged so as to maintain the order of the clauses included in the combination. When there are a plurality of main parts, the main parts are rearranged as described above.

  Next, in the partial sentence unit, the parts that are not identified as the main part by the main part identifying unit 40 are rearranged so that the order of the clauses included in the part is reversed. Note that the combination of clauses that are in parallel relation with the parallel identification unit 36 and the combination of clauses that are identified are rearranged so as to maintain the order of the clauses included in the combination.

  Next, in the partial sentence unit, for each of the phrases rearranged as described above, the word order identified by the phrase identification unit 34 is moved so that the function word included in the phrase is moved to the beginning of the phrase. Sort by. For the combination of phrases identified by the parallel identification unit 36 as a parallel relationship between phrases and the combination of the identified phrases, the function word included in the phrase at the rearmost of the combination is positioned at the forefront of the combination. Rearrange at the beginning of the phrase. Note that the specific function words may not be rearranged. For example, the case particles “ha” and “ga” may not be rearranged.

  When all the partial sentences are rearranged as described above, each of the rearranged partial sentences is connected, and the rearranged result in the word order of the sentences described in Japanese received in the input unit 10 And

  The translation unit 52 translates the input sentence described in Japanese rearranged by the rearrangement unit 50 using a known machine translator, and stores it in the model storage unit 54 from a plurality of translation candidate sentences. Based on the plurality of translation models and the weights for each of the translation models, a translation candidate sentence with the optimum translation score is selected and output to the output unit 90. The translation method is a known statistical machine of Non-Patent Document 6 (Phillip Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical Phrase-based Translation. In Proc. HLT-NAACL, pages 263-270.). It may be realized by a translation technique, and detailed description is omitted.

  The output unit 90 restores or adds the XML or HTML tag deleted in the pre-input process to the sentence described in English translated by the translation unit 52, and outputs it to the outside through an output device or a network.

<Configuration of translation model learning device>
Next, the configuration of the translation model learning device according to the embodiment of the present invention will be described. As shown in FIG. 2, the translation model learning device 200 according to the embodiment of the present invention includes a CPU, a RAM, a ROM for storing a program and various data for executing a translation model learning process routine to be described later, It can comprise with the computer which includes. Functionally, the translation model learning apparatus 200 includes an input unit 210, a calculation unit 220, and an output unit 290 as shown in FIG.

  The input unit 210 accepts a parallel learning corpus that is a set of a plurality of sentence pairs and phrase pairs (parallel translation data) that are translated into Japanese and English from an input device such as a keyboard. Note that the input unit 210 may accept input from the outside via a network or the like.

  The arithmetic unit 220 includes a learning parallel corpus 222, a model learning unit 250, and a model storage unit 270.

  The learning parallel corpus 222 stores the learning parallel corpus received by the input unit 210. The learning parallel corpus is a text file in which Japanese and English translations are written at the same number of lines.

  The model learning unit 250 includes a learning data language analysis unit 252, a learning data rearrangement unit 254, a model parallel corpus 256, a translation model learning unit 258, a translation unit 260, and a model weight learning unit 262. ing. The model learning unit 250 learns statistical translation models and model weights using the learning parallel corpus stored in the learning parallel corpus 222 and stores them in the model storage unit 270.

  The learning data language analysis unit 252 includes a morpheme analysis unit, a phrase identification unit, a parallel identification unit, a sentence division unit, and a main part identification unit (not shown) similar to the language analysis unit 30, and is included in the learning parallel corpus. Perform Japanese and English language analysis for each bilingual data in Japanese and English. Japanese language analysis is the same language analysis as the language analysis performed by the language analysis unit 30 of the translation apparatus 100 (morpheme analysis, phrase identification, phrase combination identification in parallel relation, sentence division, and main part Identification). On the other hand, English language analysis identifies word breaks. Since English is usually divided for each word, it is possible to cut out only the end-of-sentence symbol or the like, or any other word division method generally used in this field may be used.

  The learning data rearranging unit 254 is configured to perform the sorting with the rearranging unit 50 of the translation device 100 on the Japanese sentence or the phrase included in the bilingual data for each of a plurality of bilingual data analyzed by the learning data language analyzing unit 252. Similarly, the order of words is rearranged by rearranging and connecting phrases and words in the phrases in the unit of partial sentences. The learning data rearrangement unit 254 combines the rearranged sentence or phrase described in Japanese and the sentence or phrase described in English (separated into words) included in the corresponding parallel translation data. Each of them is stored in the model parallel corpus 256 as a model parallel corpus.

  The model parallel corpus 256 includes a sentence or phrase written in Japanese sorted by the learning data sorting unit 254 and a sentence or phrase written in English included in the corresponding parallel translation data (delimited by words). A parallel translation data set is stored as a parallel corpus for models.

  The translation model learning unit 258 learns a plurality of statistical translation models based on the set of parallel translation data stored in the model parallel corpus 256 and stores it in the model storage unit 270. Statistical translation models include “phrase translation model”, “phrase rearrangement model”, “language model” and the like used in statistical machine translation techniques represented by Non-Patent Document 6, and restrictions on machine translation. As appropriate, the configuration can be selected. As a learning method, Non-Patent Document 6 or the like is widely known, but the present embodiment is not limited to a specific method and can be applied.

  For example, the translation model learning unit 258 uses a statistical phrase based on bilingual data including a sentence or phrase written in Japanese and a sentence or phrase written in English stored in the model parallel corpus 256. Learn each of translation model and phrase rearrangement model. Further, a language model is learned based on sentences or phrases described in English stored in the model parallel corpus 256.

  The translation unit 260 uses a known machine translator and a plurality of translations stored in the model storage unit 270 for each Japanese sentence or phrase stored in the model parallel corpus 256 that is weight learning data. A plurality of translation candidates are created using the model and weights for each of the plurality of translation models. For example, a translation score (for example, a weighted addition value of each translation model score) is calculated using a weight for each translation model stored in the model storage unit 270, and the calculated translation score is determined from a certain value. Only translation candidates having a larger value are extracted as translation candidates.

  The model weight learning unit 262 includes a plurality of translation candidates for weight learning data, and sentences or phrases (correct translation) written in English corresponding to the weight learning data stored in the model parallel corpus 256. Based on this, a translation evaluation scale (for example, BLEU value) for each translation candidate extracted by the translation unit 260 is calculated.

  Then, based on the translation evaluation scales of the translation candidates extracted by the translation unit 260, the model weight learning unit 262 increases the translation candidate that is a better translation with respect to each of the translation models stored in the model storage unit 270. The weight for each translation model is optimized so that the translation score calculated using the weight (for example, the weighted addition value of the scores of each translation model) becomes high, and the weight for each translation model is assigned to the model storage unit 270. To remember.

  The translation candidate extraction performed in the translation unit 260 and the weight learning performed in the model weight learning unit 262 are repeatedly performed until the weights for each of the translation models converge.

  The output unit 290 outputs the translation model and model weight stored in the model storage unit 270.

<Operation of translation model learning device>
Next, the operation of the translation model learning device 200 according to the embodiment of the present invention will be described. First, the input unit 210 receives a learning parallel corpus that is a set of sentence pairs and phrase pairs (translation data) that are parallel translations of Japanese and English, and stores them in the learning parallel corpus 222. Then, when the CPU executes the program stored in the ROM of the translation model learning device 200, the translation model learning processing routine shown in FIG. 3 is executed.

  First, in step S100, the learning parallel corpus stored in the learning parallel corpus 222 is read.

  Next, in step S102, for each Japanese sentence or phrase included in the learning parallel corpus read in step S100, a known morphological analyzer (such as JUMAN or MeCab) is used to perform morphological analysis (word breaks and word breaks). Identification of part of speech).

  Next, in step S104, each sentence or phrase described in Japanese that has been subjected to morphological analysis obtained in step S102 is parsed using a known Japanese analyzer (KNP), and the parse analysis is performed. Using the information of only the clause among the results, for each clause, a classification of “content word” and “function word” is given to each word as each element in the clause.

  Next, in step S106, for each sentence or phrase described in Japanese that has been subjected to morphological analysis acquired in step S102, based on the result of syntax analysis using a known Japanese analyzer, Identify a combination of phrases.

  Next, in step S108, for each sentence or phrase described in Japanese that has been subjected to morphological analysis acquired in step S102, based on the result of syntax analysis using a known Japanese analyzer, The sentence is virtually divided to obtain a plurality of partial sentences.

  Next, in step S110, the case particle "ha" is used for each sentence or phrase described in Japanese obtained in step S102 in morpheme analysis using the result of syntax analysis using a known Japanese analyzer. ”And a phrase that conforms to the rule that includes“ ga ”are identified as the main part.

  Next, in step S112, word breaks are identified for each sentence or phrase described in English included in the parallel learning corpus read in step S100.

  Next, in step S114, the processing target acquired in step S108 for the Japanese sentence or phrase that is the processing target included in the learning parallel corpus read in step S100, in which the processing from step S102 to step S110 has been performed. For the partial sentence, the parallel relation is obtained based on the phrase information acquired in step S104, the combination of the phrases obtained in parallel in step S106, and the main part information acquired in step S110. To maintain the order of clauses for the combination of clauses, rearrange the order of each clause included in the main part, and rearrange the order of the clauses for parts different from the main part. .

  Next, in step S116, for each of the rearranged phrases acquired in step S114, the phrase is determined based on the classification of “content word” and “function word” of each word acquired in step S104. Rearrange words in the phrase so that the functional words included are sorted first.

  Next, in step S118, it is determined whether or not the processing in steps S114 to S116 has been completed for all partial sentences included in the Japanese sentence or phrase to be processed. When the processing of step S114 to step S116 is completed for all partial sentences included in the Japanese sentence or phrase to be processed, the process proceeds to step S120, and for all partial sentences, step S114 to step S116. If the process is not completed, the partial sentence to be processed is changed, and the process proceeds to step S114.

  Next, in step S120, a sentence or phrase described in Japanese that is obtained by concatenating each of the rearranged partial sentences obtained in step S116, and a sentence or phrase written in English included in the corresponding parallel translation data ( Together with the one acquired in step S116), the model parallel corpus 256 is stored in the model parallel corpus 256.

  Next, in step S122, it is determined whether or not the processing in steps S114 to S120 has been completed for all Japanese sentences or phrases included in the learning parallel corpus read in step S100. When the process of step S114 to step S120 is completed for all Japanese sentences or phrases, the process proceeds to step S124, and the process of step S114 to step S120 is terminated for all Japanese sentences or phrases. If not, the Japanese sentence or phrase to be processed is changed, and the process proceeds to step S114.

  Next, in step S124, based on a set of bilingual data composed of sentences or phrases written in Japanese and sentences or phrases written in English, stored in the model parallel corpus 256, statistical phrases Each of the translation model and the phrase rearrangement model is learned, and the language model is learned based on each sentence or phrase described in English stored in the model parallel corpus 256. Then, each learned translation model is stored in the model storage unit 270. In addition, the initial value of the weight for each translation model is stored in the model storage unit 270.

  Next, in step S126, for each sentence or phrase described in Japanese whose word order is rearranged in steps S114 to S120, a known machine translator, each translation model learned in step S124, and a model A plurality of translation candidates are created based on the translation score using the weights for each translation model stored in the storage unit 270.

  Next, in step S128, for each of the plurality of translation candidates created in step S126, a sentence or phrase written in English corresponding to the translation candidate stored in the model parallel corpus 256 (correct translation). Based on the above, a translation evaluation scale is calculated for each translation candidate. Then, based on the translation evaluation scale of each translation candidate and the translation score of each translation candidate, the weight for each translation model is optimized, the weight for each translation model is learned, and stored in the model storage unit 270.

  Next, in step S130, it is determined whether the weights for each of the translation models learned in step S128 have converged. If it has converged, the process ends. If not converged, the process proceeds to step S126.

<Operation of translation device>
Next, the operation of translation apparatus 100 according to the embodiment of the present invention will be described. First, the input unit 10 receives a plurality of translation models learned by the translation model learning device 200 and weights for each of the translation models, and stores them in the model storage unit 54. When a sentence written in Japanese is received by the input unit 10, the translation process routine shown in FIG. 4 is executed by the CPU executing the program stored in the ROM of the translation apparatus 100.

  First, in step S200, a plurality of translation models stored in the model storage unit 54 and weights for each of the translation models are read.

  Next, in step S202, similarly to step S102, a known morphological analyzer (JUMAN, MeCab, etc.) is used for the input sentence described in Japanese accepted by the input unit 10, and morphological analysis ( Word separation and part-of-speech identification).

  Next, in step S204, as in step S104, the input sentence analyzed in step S202 is parsed using a known Japanese analyzer (KNP), and the result of the parse analysis is analyzed. Among them, using only the clause information, for each clause, a classification of “content word” and “function word” is given to each word as each element in the clause.

  Next, in step S206, as in step S106, the combination of clauses that are in a parallel relationship is identified based on the result of the syntax analysis acquired in step S204.

  Next, in step S208, as in step S108, based on the result of the syntax analysis acquired in step S204, the sentence is virtualized with punctuation based on the result of syntax analysis using a known Japanese analyzer. To obtain a plurality of partial sentences.

  Next, in step S210, as in step S110, the case particles “ha” and “ga” are based on the result of the syntax analysis acquired in step S204 in the unit of the sentence-segmented partial sentence acquired in step S208. The phrase that conforms to the rule of the phrase including is identified as the main part.

  Next, in step S212, for the partial sentence to be processed in the input sentence acquired in step S208, the phrase information acquired in step S204, the combination of phrases acquired in parallel in step S206, and the step Based on the information of the main part acquired in S210, the order of the clauses included in the main part is rearranged so as to be reversed so as to maintain the order of the clauses for the combination of clauses in parallel relation. Rearrange parts so that the order of each phrase is reversed.

  Next, in step S214, for each of the rearranged phrases acquired in step S212, the phrase is determined based on the classification of “content word” and “function word” of each word of each phrase acquired in step S204. Rearrange words in the phrase so that the functional words included are sorted first.

  Next, in step S216, it is determined whether or not the processing in steps S212 to S214 has been completed for all partial sentences included in the input sentence. When the process from step S212 to step S214 is completed for all partial sentences included in the input sentence, the process proceeds to step S218, and the process from step S212 to step S214 is not completed for all partial sentences. In this case, the partial sentence to be processed is changed, and the process proceeds to step S212.

  Next, in step S218, each of the rearranged partial sentences acquired in step S214 is connected.

  Next, in step S220, the input sentence obtained by rearranging the word order acquired in step S218 is translated using a known machine translator, and a plurality of translation models acquired in step S200 from a plurality of translation candidate sentences. And a translation candidate sentence with an optimal translation score is selected based on the weight for each of the translation models.

  Next, in step S222, the translation result selected in step S220 is output and the process ends.

<Example of Translation Model Learning Device 200>
Next, an embodiment in which a statistical translation model is learned from a Japanese-English parallel corpus of about 3 million sentences will be described below.

  The learning data language analysis unit 252 of the translation model learning device 200 performs language analysis of Japanese and English. In the Japanese language analysis, the same processing as the language analysis unit 30 of the translation apparatus 100 is performed, and the same result is obtained. In English language analysis, the word segmentation program bundled with Moses is used, and only word segmentation is performed.

  In the learning data rearrangement unit 254 of the translation model learning device 200, the same sort processing as that of the rearrangement unit 50 of the translation device 100 is performed for the Japanese of the parallel corpus.

  The parallel corpus of Japanese and English is rewritten into a sorted set of Japanese word strings and a set of English word strings. Each set of word strings is a text file in which one sentence is stored in one line separated by single-byte spaces for each word. In the translation model learning, a phrase program model DB: phrase-table.gz, phrase rearrangement model DB: from a text file representing a set of Japanese and English word strings by a learning program provided by Moses. reordering-table.wbe-msd-bidirectional-fe.gz ".

  Further, “language model DB: ja.5gram.arpa.gz” is created from a text file of a set of English word strings by a known language model learning program SRILM. In this embodiment, a word 5-gram language model is created.

  Furthermore, a known method called “Minimum Error Rate Training: MERT” for determining the optimum value of the weight between models (Non-patent Document 7: Franz J. Och. 2003. Minimum error rate training for statistical machine translation.In Proceedings of the 41st annual conference of the Association for Computational Linguistics, pp.160--167, 2003.) Export to

<Example of Translation Device 100>
An example in which a Japanese sentence “a data storage device and a data collection device is connected to a network” is input to a computer terminal on which the translation device 100 is mounted will be described below.

  In this embodiment, known Japanese morphological analysis software JUMAN and known syntax analysis software KNP are used for Japanese language analysis. In the present embodiment, in order to collectively identify morphological analysis, phrase identification, and combination of phrases in parallel relation, Japanese sentences input from the input unit 10 are input from the standard input of the computer terminal to JUMAN. The output is directly input to the KNP. The KNP analysis result is output as shown in FIG. In FIG. 5, the information that is not used in the present embodiment is partly omitted.

  The symbol “*” at the beginning of a line indicates the beginning of a phrase, and the subsequent numbers indicate a related phrase (0-started phrase number) and an engagement type (D is normal modification, P is parallel). The symbol “+” at the beginning of a line indicates that the word indicated in the following line is a content word, and the word without a line starting with “+” in the previous line indicates a function word. The interpretation of the analysis result is as shown in FIG. In FIG. 6, since dependency information is not used in the present embodiment, only the combination of clauses / content words / function words in a phrase / parallel relationship is shown.

  Further, in sentence division by the sentence division unit 38, if Japanese punctuation marks are handled as division points, the input sentence is divided as shown in FIG. 7, and two partial sentences are acquired.

  Then, in the main part identification by the main part identification unit 40, a rule of “a phrase including a case particle“ ga ”in a function word” is applied, and as shown in FIG. Together with “partial sentence 1-sentence 0”, which is a parallel relationship, the main part is identified. The other parts are identified as predicates.

  As described above, the rearrangement unit 50 of the translation apparatus 100 rearranges the Japanese sentence clauses. In the present embodiment, the main part and the predicate of each partial sentence of the Japanese sentence are rearranged so that the order of the respective clauses is reversed except for the combination of the clauses having a parallel relation. FIG. 9 shows the result of rearranging the phrases.

  Next, for each clause for each partial sentence of the Japanese sentence in which the above clauses are rearranged, the function word is moved to the head of the relevant sentence. Here, for the combination of clauses that have a parallel relationship, among the combinations of clauses that have a parallel relationship, the function word of the clause that is positioned last is the head of the clause that is positioned at the beginning of the combination of clauses that have a parallel relationship The rule of moving to is used. FIG. 10 shows the result of rearrangement.

  Finally, by combining the sorted sub-sentences, a sorted word string “also connected to the data storage device and the data collection device connected to the network” is obtained.

  The translation unit 52 of the translation apparatus 100 translates the Japanese sentences rearranged by the rearrangement unit 50 into English. In this embodiment, the statistical translation model (phrase translation model, phrase rearrangement model, language model) learned by the translation model learning unit 258 of the translation model learning device 200 and the weights for each model are used, and known statistical translation software is used. Moses was used. In the present embodiment, the output result of Moses “then, the data storage device and a data collecting device are connected to the network” is output to the output unit 90.

  It has been found that the machine translation program realized by the method of the above embodiment shows higher translation performance than the machine translation program configured by the prior art and the machine translation program based on the conventional Japanese rearrangement method. In an experiment using a statistical model learned with a Japanese-English parallel corpus of about 3 million sentences, the values of the evaluation indexes BLEU and RIBES that are most commonly used in the field are the conventional techniques described in Non-Patent Document 4 above. The configured machine translation programs were 31.44 and 72.88, respectively, whereas the machine translation program of this embodiment achieved 31.84 and 74.28.

  As described above, according to the translation apparatus according to the embodiment of the present invention, the order of each phrase in the main part is rearranged so that the order of each phrase in the part different from the main part is inverted. Thus, the function words of each clause are rearranged at the head of the clause, and the rearranged sentences are translated, so that the translation can be performed with high accuracy.

  Also, in the translation from Japanese to English, it is possible to rearrange the Japanese word order to the word order close to English, and as a result, the difference in word order is very small in the translation from Japanese to English. , Making translation easier.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, an input sentence written in Japanese has been described for rearranging the word order so that it resembles the English word order, but the present invention is not limited to this, and input sentences in other languages are used. Also good. In this case, the word order may be rearranged so that the input sentence of the first language is similar to the word order of the second language different from the first language.

  Moreover, in this Embodiment, although the translation apparatus demonstrated the case where it translated with respect to an input sentence, it is not this limitation. For example, the translation device may translate the input phrase.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium or provided via a network. It is also possible to do.

DESCRIPTION OF SYMBOLS 10 Input part 20 Arithmetic part 30 Language analysis part 32 Morphological analysis part 34 Phrase identification part 36 Parallel identification part 38 Sentence division part 40 Main part identification part 50 Rearrangement part 52 Translation part 52 Model storage part 90 Output part 100 Translation apparatus 200 Translation Model learning device 210 Input unit 220 Operation unit 222 Learning parallel corpus 250 Model learning unit 252 Learning data language analysis unit 254 Learning data rearrangement unit 256 Model parallel corpus 258 Translation model learning unit 260 Translation unit 262 Learning unit 270 Model storage unit 290 output section

Claims (7)

For an input sentence or phrase, a phrase identification unit for identifying each phrase including a function word;
A main part identifying unit for identifying the main part of the input sentence or phrase;
The main part of the input sentence or phrase identified by the main part identification part is rearranged so as to reverse the order of each phrase included in the main part, and the main part of the input sentence or phrase is For different parts, the order of each phrase is reversed, and the function words included in the phrase for each phrase sorted for the main part and for each phrase sorted for a part different from the main part A rearrangement unit that rearranges the word order in the clause so that
Including a word order rearrangement device. A parallel identification unit for identifying a combination of clauses in parallel relation from each clause of the input sentence or phrase;
The rearrangement unit rearranges the order of the clauses included in the main part so as to reverse the order of the clauses with respect to the combination of clauses identified in parallel by the parallel identification unit. , Rearrange the order of the clauses in the part different from the main part so that the order of the clauses is reversed, and for the combination of the clauses in the parallel relationship, the function word included in the clause located at the rearmost position Reordering at the beginning of a clause, and reordering the word order within the clause so that the function words included in the clause are reordered at the beginning of the clause for each clause that is different from the combination of clauses in the parallel relationship The word order rearrangement device according to Item 1. In a translation apparatus for translating an input sentence or phrase described in a first specific language into a sentence or phrase described in a second specific language different from the first specific language,
About the result of rearranging the input sentence or phrase by the word order rearrangement device according to claim 1 or 2, in the second specific language based on weights for each of a plurality of types of translation models and the plurality of types of translation models. A translation device including a translation unit that translates a written sentence or phrase. A word order rearrangement method in a word order rearrangement device including a phrase identification unit, a main part identification unit, and a rearrangement unit,
The phrase identifying unit identifies each phrase including a function word for an input sentence or phrase,
The main part identifying unit identifies the main part of the input sentence or phrase;
The reordering unit reorders the main part of the input sentence or phrase identified by the main part identifying part so as to reverse the order of each clause included in the main part, The parts different from the main part are rearranged so as to reverse the order of the clauses, the clauses rearranged for the main part and the clauses rearranged for the parts different from the main part, The word order rearrangement method of rearranging the word order in the clause so that the functional words included in the clause are rearranged at the head of the clause. A translation method in a translation device for translating an input sentence or phrase described in a first specific language including a translation unit into a sentence or phrase described in a second specific language different from the first specific language,
The result of rearranging the input sentence or phrase by the word order rearranging method according to claim 4 based on a plurality of types of translation models and a weight for each of the plurality of types of translation models. A translation method that translates sentences or phrases written in a specific language.   The program for functioning a computer as each part which comprises the word order rearrangement apparatus of Claim 1 or Claim 2.   The program for functioning a computer as each part which comprises the translation apparatus of Claim 3.