JP5439776B2 - Word alignment device and word alignment program - Google Patents

Description

  The present invention relates to a word alignment apparatus and a word alignment program, and more particularly, to a word alignment example parallel translation dictionary learning apparatus and an example sentence base machine translation apparatus for extracting translated words.

  Machine translation is the conversion from one language to another using a computer, and such research and development has been conducted all over the world for half a century. Machine translation systems can be broadly divided into 1) analysis-based machine translation systems, 2) statistical-based machine translation systems, and 3) example sentence-based machine translation systems.

  The analysis-based machine translation system is a technique for performing analysis of a first language (morpheme analysis, syntax / semantic analysis, etc.), converting the analysis result to a second language, and further generating a translation of the second language. Since natural language analysis technology is still immature, there are limits to the practical application of analysis-based machine translation systems. Furthermore, since it cannot be learned, it has the disadvantage that it becomes difficult to improve and improve the translation engine.

  The statistical-based machine translation method is a technique for constructing a translation model using a language model and a statistical model. This method is limited in practical use because learning data (corpus) necessary for the configuration of each model is limited.

  The example-based machine translation method is to translate a new document by referring to a translation example sentence that has already been learned by imitating a mechanism in which a human learns a foreign language. This translation system was proposed in the 1980s, and research and development have been actively conducted since then. Example sentence-based machine translation technology includes translation memory translation technology, translation technology that uses bilingual example sentences with word alignment, translation technology that uses sentence patterns, etc. There is.

  FIG. 1 is a diagram illustrating an example of the overall configuration of a machine translation system according to an example sentence-based machine translation system. The machine translation system 10 is configured to shift from a simpler translation to a more complicated translation, thereby speeding up translation. In addition, the machine translation system 10 has a learning function that automatically collects portions that cannot be translated and gives an accurate parallel translation.

  The machine translation system 10 translates a sentence-by-sentence sentence input from the source language text sentence input unit 12, an input sentence that cannot be collated by the translation memory translation apparatus 14, that is, an input that is inappropriate. Example sentence pattern-based translation device 16 for inputting a sentence and translating a word string of the input sentence subjected to morphological analysis, and an input sentence that could not be translated by example sentence pattern-based translation apparatus 16 as an inappropriate sentence, as a result of the morphological analysis A word translation unit 18 that inputs a word string and translates the word string, and a target language text output unit 20 that generates a text sentence in a target language based on the result of appropriate translation by the translation unit and outputs it. And have.

  Further, the machine translation system 10 collects a sentence that could not be translated by the example sentence pattern-based translation device 16, and creates an appropriate translation for the collected sentence. A learning device 24 for checking and correcting the translation created by the automatic sentence collection unit 22 and a translation dictionary 26 are provided. The translation dictionary 26 stores a word bilingual dictionary 26a that stores words in the first language and a second language word that is a translation thereof, and an example sentence in the first language and an example sentence in the second language that is a translation thereof. The example sentence parallel translation dictionary 26b includes an example sentence pattern parallel translation dictionary 26c that stores example sentence patterns in the first language and example sentence patterns in the second language that are the translations of the first language example patterns. The translation dictionary 26 is used in the translation memory translation device 14, the example sentence pattern base translation device 16, and the word direct translation translation device 18. Note that the machine translation system of FIG. 1 is an example of a configuration, and there is also a system that includes another translation engine.

  FIG. 2 is a diagram for explaining the example sentence pattern-based translation apparatus 16 shown in FIG. 1, and shows an example sentence base translation apparatus using a word alignment example sentence parallel translation dictionary. This example sentence base translation apparatus performs translation using a word alignment example sentence parallel translation dictionary, and translates example sentences that are similar to an input sentence with high accuracy. As shown in the figure, a Chinese input sentence is subjected to morphological analysis, and a similar example sentence is searched from a word alignment example sentence parallel translation dictionary. Then, the difference between the input sentence and the example sentence and the correspondence relationship are calculated, and a Japanese translation is generated using the word parallel translation dictionary.

  There are several reports on these translations. Patent Document 1 discloses a technique for extracting a translated word of a second language corresponding to an expression of the first language from a sentence of the second language based on the similarity of pronunciation. Patent Document 2 relates to a translation pair extraction device that extracts translation pairs from a parallel text corpus, and estimates the phonemes of words in the first language and the second language, and outputs words that match both phonemes as word pairs. is there. Non-Patent Document 1 extracts an example sentence and a word constituting the translated sentence by performing morphological analysis on the example sentence and the translated sentence, respectively, for the example sentence pair including the example sentence of the first language and the translated sentence of the second language of the example sentence. A technique for extracting translations of words constituting an example sentence from a word bilingual dictionary and collating the extracted translations with a translation word is reported. In Non-Patent Document 2, alignment is performed using a statistical model between a word and a translation word, which is composed of a set of example sentence pairs including an example sentence in the first language and a translation sentence in the second language of the example sentence. For example, the statistical model is a DICE coefficient, X2, mutual information, T-score, and the like.

  Patent Document 3 discloses a word alignment example sentence parallel translation dictionary learning device and a word extraction device that can perform word alignment with high recall rate and high accuracy.

Japanese Patent Laid-Open No. 10-143514 Japanese Patent Laying-Open No. 2005-258637 JP 2007-199793 A Jin-Xia Huang, Key-Sun Choi. 2000.Using Bilingual Semantic Information in Chinese-Korean Word Alignment.Pacfic Asia Conference on Language, Information and Computation.PACLIC14, pp121-130. Melamed, Dan. “A Word-to-Word Model of Translational Equivalence”. In Procs. Of the ACL97.pp 490-497. Madrid Spain, 1997.

  The above-described example sentence-based translation device has a function of automatically extracting and assigning a correspondence relationship between a word and its translated word between the first language example sentence and the second language translation of the example sentence. Required.

  FIG. 3 is a diagram for explaining alignment between an example sentence and words included in the translation. FIG. 3A shows a pair of input sentences in Chinese and Japanese of the translation, and FIG. An ideal alignment between Chinese words and Japanese words obtained by morphological analysis of these input example sentence pairs is shown. In the relationship between all example sentences and translated sentences, it is very unrealistic to set accurate word alignment as shown in FIG. 5B by human manual work. On the other hand, in order to automatically generate alignment between words, a high recall rate and high accuracy are required. The recall rate is the ratio between the number of pairs of words that should be extracted from the example sentence and the translation and the number of pairs of words that are actually extracted, and the accuracy is the number of pairs of words that are actually extracted and the number of correct words. It is the ratio to the number of pairs and is shown in the following equation.

  Regarding word alignment, Patent Document 3 utilizes a bipartite graph optimal matching method. FIG. 4 shows the overall configuration of the word alignment apparatus based on the bipartite graph optimal matching. The word alignment 100 includes an input unit 102 for inputting an example sentence and a translation example pair of the translated sentence, an output part 104 for outputting the alignment result, a morphological analysis of the example sentence and the translated sentence, and extracting a word and a part of speech included in the morphological analysis. The preprocessing unit 106 to be assigned, the example sentence bilingual dictionary 108 for storing pairs of example sentences and their translations, the example sentences and the translation sentences inputted from the example sentence bilingual dictionary 108, respectively, extract the example sentences and the translation words and parts of speech respectively. Constructed by a corpus pre-processing unit 110 that constructs a corpus index that can be searched, a word bilingual dictionary 112 that stores words and their translations, a similarity calculation unit 114 that performs word alignment using a word bilingual dictionary, and a corpus pre-processing unit 110 A word index table 116 that stores a word index table, and corpus statistical information of the word index table is used. Correlation degree calculation unit 118 for obtaining the degree of correlation between a word and a translated word, a bipartite graph base that optimizes alignment between words by solving alignment candidates with low confirmation and multiple correspondence alignment problems by a bipartite graph matching method An alignment unit 120 is included.

  As shown in FIG. 5 (a), the similarity calculation unit 114 and the correlation calculation unit 118 calculate the dictionary similarity between the words in the upper first language and the words in the lower second language, and the meaning between the words. The reliability is calculated from the above similarity, the shape similarity between words, the part-of-speech similarity, the correlation between words and translations, etc., and the weights of each branch are calculated with the calculated values. The bipartite graph base alignment unit 120 obtains the word alignment optimized on a one-to-one basis as shown in FIG.

However, such a word alignment method has the following three problems.
1) When there is no accurate translation in the word parallel translation dictionary 112 shown in FIG. In FIG. 6A, since there is no accurate Japanese translation of the Chinese word, the Japanese explanation translation is used as a parallel translation. On the contrary, in FIG. 6B, since there is no accurate Chinese translation of the Japanese word, the Chinese explanation translation is used as a parallel translation. When such a word parallel translation dictionary 112 is used, the coverage of the alignment method using the dictionary similarity SimD in the similarity calculation unit 114 shown in FIG. 4 is adversely affected.

  2) A word segmentation device in a machine translation system, particularly a Chinese word segmentation device (such as morpheme analysis), tends to take a large unit of word (length of a character string) in order to improve translation accuracy. When the word unit is increased, the accuracy of translation increases, but inconsistency occurs between the word in the word bilingual dictionary and the word obtained by the word segmentation device. For example, when the word obtained by the word segmentation device includes the character string “W1W2”, the word bilingual dictionary does not include a word including the character string “W1W2”. Instead, “W1”, There are two individual words “W2”.

  3) The matching alignment method with the maximum and minimum weights by the bipartite graph is very effective for one-to-one word alignment, but is not necessarily an effective method for many-to-many word alignment.

The present invention solves such conventional problems, and an object of the present invention is to provide a word alignment apparatus and a word alignment program in which the conventional word alignment technique is improved and the recall rate and accuracy are improved.
It is another object of the present invention to provide a word alignment apparatus and a word alignment program having a one-to-many, many-to-one, and many-to-many word alignment function.
A further object of the present invention is to provide a word alignment example translation dictionary and a word translation dictionary using word alignment technology, and provide a translation device and a translation support system using these dictionaries.

  A word alignment apparatus according to the present invention performs alignment between words included in a first language example sentence and a second language example sentence that is a translation of the example sentence, and includes a first language word and its translation A second language word, a second language translation for the first language word, a word bilingual dictionary that stores the correspondence of the first language translation for the second language word, and a first language example sentence And an input means for inputting a second language example sentence which is a translation of the example sentence, and a word extracting means for extracting a first language word and a second language word from the inputted first language and second language example sentences, respectively. When the correspondence relationship between the extracted first language word and the extracted second language word is included in the word parallel translation dictionary, the extracted first language word is the extracted first language word. Alignment to bilingual words The word bilingual dictionary includes the correspondence between the first language word and the second language translation of the first language word that is not aligned by the first alignment means and the first language word. A second alignment means for aligning words in the first language with the extracted plurality of words in the second language that are common to the translated words in the second language.

  Preferably, the word alignment apparatus further divides the first language word for the first language word that is not aligned by the second alignment means, and the correspondence relationship between the divided word and the second language translation is When included in the word bilingual dictionary, there is provided a third alignment means for aligning the divided words with the plurality of extracted second language words that are common to the second language translation words.

  Preferably, the word alignment device further includes a first language word that is not aligned by the third alignment means when the first language word and the extracted second language word match, And a fourth alignment means for aligning one language word with the extracted second language word. For example, the notation match is a match between the kanji characters of the first language word and the second language word.

  Preferably, the first and fourth alignment means convert from one word in the first language to a plurality of consecutive words in the second language, or from one word in the second language to a plurality of consecutive words in the first language. Align.

  Further, the word alignment apparatus according to the present invention performs alignment between words included in each example sentence in the first language and each example sentence in the second language that is a parallel translation of the example sentence. Storage means for storing a part-of-speech connection table showing a connection relation between parts of speech of two consecutive words using information that has been pre-aligned with respect to words between the two languages, an example sentence in the first language, and the example sentence Input means for inputting example sentences in the second language as a parallel translation; word extraction means for extracting words in the first language and the second language from the inputted example sentences in the first language and the second language; When a word in the first language is aligned with a part of the extracted plurality of consecutive words in the second language and the remaining words are not aligned, the plurality of consecutive words are the part of speech. If included in the connection table has an alignment means for aligning the word of the first language as the extracted plurality of consecutive words of the second language.

  Preferably, the alignment means aligns a part of the extracted consecutive words to the extracted second language word and the remaining words are not aligned when the remaining words are not aligned. Is included in the part-of-speech connection table, the extracted consecutive first language words are aligned with second language words.

  Preferably, the part-of-speech connection table uses a corpus to which word alignment is assigned, the number of all two consecutive words included in the corpus, and two consecutive words aligned in the corpus. A part-of-speech connection with a certain frequency of occurrence is specified from a number ratio.

  Furthermore, the word alignment apparatus according to the present invention performs alignment between words included in each example sentence in the first language and each example sentence in the second language that is a translation of the example sentence, Input means for inputting an example sentence in a second language that is a translation of the example sentence; word extraction means for extracting words in the first language and the second language from the inputted example sentences in the first language and the second language; An alignment dictionary that stores correspondence between one word in one language and a plurality of non-consecutive words in the second language that are the translations thereof, and the extracted first language word are included in the alignment dictionary. And when the second language example sentence includes a plurality of non-consecutive words in the second language that are parallel translations of the words in the first language, the extracted words in the first language are used as the words in the second language. Applies to multiple non-consecutive words To Imento.

  Furthermore, the word alignment apparatus according to the present invention performs alignment between words included in each example sentence in the first language and each example sentence in the second language that is a translation of the example sentence, Input means for inputting an example sentence in a second language that is a translation of the example sentence; word extraction means for extracting words in the first language and the second language from the inputted example sentences in the first language and the second language; From the example sentence parallel translation dictionary storing the example sentence of one language and the translation of the second language as its translation, the pair of the extracted example sentence including the plurality of consecutive words of the first language and the translation thereof is selected and selected. Extracting a set of common parts between the translated example sentence parallel translation pairs, calculating a support degree of the extracted common part for the plurality of continuous words, and calculating the plurality of continuous parts based on the calculated support degree The second language And a alignment means for aligning the plurality of words is a common part.

  Preferably, the degree of support is calculated by selecting a common part of the corresponding translated sentence in the second language as an alignment candidate when the common part of each set has a predetermined matching degree with respect to the plurality of consecutive words. And means for counting the number of appearances of the selected alignment candidate as representing the degree of support.

  Preferably, when the number of appearances of the alignment candidate with the highest number of appearances exceeds the first threshold value, or the appearance of the translation candidate with the number of appearances equal to or less than the first threshold value and the next most frequent appearance number The difference between the number of occurrences exceeds the second threshold value, or the ratio of the number of appearances to the first threshold value and the number of appearances and the number of appearances of the translated sentence candidate having the next highest number of appearances is the third threshold value. In any case of exceeding the above, the alignment candidate with the largest number of appearances is set as the alignment of the plurality of consecutive words.

  An example sentence base translation apparatus according to the present invention is input with reference to the above word alignment apparatus, an example sentence bilingual dictionary storing example sentences in a first language and an example sentence in a second language which is a translation thereof, and the example sentence parallel translation dictionary. A translation generation unit that generates a second language translation of the first language example sentence, and the translation generation unit generates a translation using an alignment result of the word alignment device.

  The word alignment program according to the present invention is executed by a word alignment apparatus that performs alignment between words included in each example sentence in the first language and each example sentence in the second language that is a parallel translation of the example sentence. Inputting a language example sentence and a second language example sentence that is a translation of the example sentence; extracting first and second language words from the inputted first language and second language example sentences; When the correspondence relationship between the extracted first language word and the extracted second language word is included in the word bilingual dictionary, the extracted first language word is converted to the extracted second language. A first alignment step that aligns the first language word, and a first language word that is not aligned by the first alignment step, and the second language word and second A second word that aligns a first language word with a plurality of extracted second language words that are in common with a second language translation word when the word bilingual correspondence is included in the word bilingual dictionary And the first language word that is not aligned by the second alignment step, the first language word is divided, and the correspondence between the divided word and the second language translation is a word bilingual dictionary A third alignment step of aligning the divided words with the extracted second language words that are in common with the second language translation.

  Furthermore, the word alignment program according to the present invention is executed by a word alignment apparatus that performs alignment between words included in each example sentence in the first language and each example sentence in the second language that is a translation of the example sentence. A step of inputting an example sentence in one language and an example sentence in a second language that is a translation of the example sentence, and a step of extracting words in the first language and the second language from the inputted example sentences in the first language and the second language, respectively And when the extracted first language word is aligned with some of the extracted second language consecutive words and the remaining words are not aligned, the consecutive words Is included in the part-of-speech connection table indicating the connection relationship between the parts of speech of two consecutive words prepared in advance, the extracted words in the first language are used as consecutive words in the second language. And a step of aligning the plurality of words.

  In the word alignment program, the extracted first language word is further included in an alignment dictionary that stores correspondence between one word in the first language and a plurality of non-consecutive words in the second language, which is a translation of the word. And when the second language example sentence includes a plurality of non-consecutive words in the second language that are parallel translations of the words in the first language, the extracted words in the first language are The word alignment program according to claim 16, further comprising the step of aligning with a plurality of non-contiguous words of a language.

  Preferably, the word alignment program further includes a step of selecting, from the example sentence parallel translation dictionary, a pair of the extracted example sentences including a plurality of consecutive words in the first language and the parallel translation, and between the selected example sentence parallel translation pairs Extracting a set of common parts, calculating a support level of the extracted common parts for the plurality of continuous words, and determining the plurality of continuous words based on the calculated support level. Aligning a plurality of words that are common parts of the second language.

  According to the present invention, since one-to-many, many-to-one, or many-to-many alignment is possible between words in the first language and words in the second language, the recall rate and accuracy can be improved as compared with the prior art. Can do. Further, even when the word bilingual dictionary does not include an accurate parallel translation but includes a commentary translation, the coverage rate can be improved. Furthermore, by dividing the unaligned word, the coverage of the word bilingual dictionary can be improved.

  According to the present invention, a high-performance word alignment method can be expected, and the use of linguistic information, similar information, statistical information, and the optimal matching method between bipartite graphs enables high recall rate and high-precision word alignment. An example sentence parallel dictionary learning engine can be realized.

  Furthermore, according to the present invention, the efficiency of research and development can be improved. For example, the word alignment example sentence parallel translation dictionary and the word parallel translation dictionary can be automatically generated from the example sentence parallel translation dictionary, so that the efficiency of constructing the parallel translation dictionary necessary for the translation system can be improved. This makes it possible to reduce the cost of constructing a bilingual dictionary, shorten the construction time, and automate construction.

The best mode for carrying out the present invention will be described with reference to the embodiments shown in the drawings.
The features of the word alignment apparatus of the present embodiment are as follows.
1) Even if the word bilingual dictionary uses an explanatory translation of a word as a translation, it enables word alignment using the dictionary similarity SimD.
2) Even if a word obtained by a word segmentation device (for example, a morphological analysis device) does not exist in the word parallel translation dictionary, word alignment using the dictionary similarity SimD is enabled.
3) Enable many-to-many word alignment.

Many-to-many word alignment types are classified into the following six types.
S = CS <-> JS represents one parallel example sentence pair. Here, CS is an example sentence in the first language, and JS is a translation in the second language of CS. CS = <C1, C2, ..., Cm>, JS = <J1, J2 ..., Jn>. Here, Ch and Jk are words.
1) 1-n continuous type: Ch is Jh, Jh + 1, Jh + 2,. . . Align with n consecutive words of.
2) m-1 continuous type: m consecutive words of Ch, Ch + 1, Ch + 2, ... align with Jy.
3) mn continuous type: Ch, Ch + 1, Ch + 2,... M consecutive words are Jh, Jh + 1, Jh + 2,. . . Align with n consecutive words of.
4) 1-n non-continuous type: Ch is Jh, Jx, Jy,. . . Align with n non-contiguous words.
5) m-1 discontinuous type: m discontinuous words of Ch, Cs, Ct, ... align with Jy.
6) mn non-continuous type: m discontinuous words of Ch, Cs, Ct,. . . Align with n non-contiguous words.

  In this embodiment, the sixth m-n non-continuous alignment is excluded. This is because there are very few examples of this type. In the following embodiment, an example of Chinese and Japanese word alignment will be described.

  FIG. 7 is a block diagram showing the configuration of the word alignment apparatus of the present embodiment. The word alignment translation apparatus 200 according to the present embodiment includes an input unit 202 that inputs an example sentence in the first language and an example sentence in the second language that is a translation thereof, a morpheme analysis part 204 that analyzes the input example sentence, and a morpheme analysis Pre-processing unit 206 that pre-processes a character string or a word, correspondence relationship between a first language word and a second language word that is the translated word, a second language word and a first language word that is the translated word Refer to the word bilingual dictionary 208 and the word bilingual dictionary 208 that store the correspondence, the second language explanation translation for the first language word, and the first language explanation translation for the second language word. The dictionary base alignment unit 210 that performs alignment between the first language word and the second language word, the Chinese simplified conversion unit 212 that performs simplified Chinese character conversion, and the Chinese simplified conversion unit 212 conversion A word notation base alignment unit 214 that performs alignment of word notation using results, a storage unit (memory) 216 that stores information on various alignments described later, and a multi-pair using alignment information stored in the storage unit 216 It includes a many-to-many alignment unit 218 that performs multiple word alignment, a BGMWM base alignment unit 220 that performs bipartite graph-based alignment, and an output unit 222 that outputs alignment results between words.

  The word alignment apparatus 200 of the present embodiment is preferably included in an example sentence base translation apparatus 16 as shown in FIG. The example sentences that could not be properly translated in the translation memory translation apparatus 14 shown in FIG. 1 are input to the example sentence base translation apparatus 16. As shown in FIG. 2, the input example sentence is compared with a word alignment example sentence parallel translation dictionary, and an example sentence pattern in which an example sentence similar to the example sentence and its translation word are paired is extracted. Preferably, such an example sentence parallel translation pattern is input to the input unit 202.

  The morpheme analysis unit 204 performs morphological analysis on the inputted example sentence pattern to separate words constituting the example sentence and the translated sentence, and gives the part of speech of the word. The technology of morphological analysis is widely known. For example, Chasen for Japanese morphological analysis technology, Seg and POS tool of Tsinghua University and CiPosSDK tool of Tohoku University of China for Chinese morphological analysis technology. Can be used. FIG. 8 shows an example of Chinese morphological analysis and an example of Japanese morphological analysis.

  FIG. 9 shows an operation flow of the preprocessing unit. The preprocessing unit 206 receives the word and part-of-speech ID (see FIG. 8) cut by the morphological analysis unit 204 (step S101), and extracts a numerical part from the word (step S102). Next, the numerical value portion and the non-numeric value portion are separated, the numerical value portion is expressed in Chinese characters (steps S103 and S104), and the numerical value portion expressed in Chinese characters and the non-numeric value portion are synthesized (step S105). Thereby, alphabets and numbers are converted into Kanji expressions.

  Next, the dictionary base alignment unit will be described. S = CS <-> JS represents one translated example sentence pair. Here, CS is an example sentence in the first language, and JS is a translated sentence in the second language of CS. Therefore, CS = <C1 C2... Cm>, JS = <J1 J2... Jn>, and Ch and Jk are words.

  The dictionary base alignment unit 210 refers to the word bilingual dictionary 208 and executes the following algorithm.

1) DBA1 algorithm: For all word pairs (Cx, Jy) where SimD (Cx, Jy) = 1, align to the words Cx and Jy.
2) DBA2 algorithm: For an unaligned word Ch, a set of all translated words of Ch in the word bilingual dictionary 208 is Dict (Ch) = {Jh1, Jh2,..., Jhx}. If there is a JS partial character string <Jk, Jk + 1, ..., Jk + y> that satisfies the following condition 1 and condition 2, the partial character strings <Jk, Jk + 1, ... of the words Ch and JS , Jk + y>.

  3) DBA3 algorithm: If there is no Ch translation in the word bilingual dictionary 208 for the unaligned word Ch, the word Ch is used by using the first language word set BiWordSet1 in the word bilingual dictionary 208. Split. The divided result is Ch = <Ch1, Ch2, ..., Chz>. Let Dict (Chi) = {Jhi1, Jhi2,..., Jhix} be the set of all translations of Chi in the word bilingual dictionary 208. If there is a JS word Jk that satisfies the following conditions 1 and 2, alignment is performed with the words Ch and JS.

  Next, the word notation base alignment unit will be described. S = CS <-> JS represents one parallel example sentence pair. CS is an example sentence in the first language, and JS is a translation in the second language of CS. CS = <C1, C2,... Cm>, JS = <J1, J2,... Jn>, and Ch and Jk are words.

  The word notation base alignment unit 212 refers to the simplified Chinese characters by the simplified Chinese character conversion unit 212, and for all unaligned words Ch, the shape similarity SimM (Ch, Jy) = 1, and If the word Jy is not aligned, it is aligned with the words Cx and Jy. SimM (Ch, Jy) = 1 is, for example, “China” and “China” when the Chinese characters have the same shape.

  Next, the many-to-many alignment unit will be described. As shown in FIG. 10, the many-to-many alignment unit 218 includes a 1-n continuous type and m-1 continuous type alignment unit 230, a 1-n non-continuous type and an m-1 non-continuous type alignment unit 232, And a continuous alignment unit 234 (n and m are natural numbers of 2 or more).

  The n-continuous type and m-1 continuous-type alignment unit 230 performs alignment of n words continuous with one word, or alignment of m consecutive words with one word.

  The n-continuous type and m-1 continuous type alignment unit 230 uses the word alignment corpus in advance to perform the part-of-speech connection table of two consecutive words for the 1-n continuous type and the m-1 continuous type alignment. Construct CPPS (Concomitance POS Pair Set). For example, when aligning to words J1, J2, and J3 in which word C1 is continuous, the part-of-speech connection between J1 and J2 and J2 and J3 is the object of calculation. Conversely, when consecutive words C1, C2, and C3 are aligned with word J1, the part-of-speech connections between C1 and C2, and C2 and C3 are subject to calculation. CPPS can be obtained by the following equation. Here, the denominator of CPPS is the number of all two consecutive parts of speech in the corpus and the numerator is the number of aligned parts of speech in the corpus. Further, θ is a threshold value, and 0 <θ <1.

  The 1-n continuous type and m-1 continuous type alignment unit 230 performs the following alignment using the part-of-speech connection table.

  S = CS <-> JS represents one parallel example sentence pair. Here, CS is an example sentence in the first language, and JS is a translated sentence in the second language of CS. CS = <C1, C2,..., Cm>, JS = <J1, J2,..., Jn>, Ch and Jk are words.

For all aligned words Jk
Case 1: If the word Ch is aligned to Jk and Jk + 1 is also not aligned, if (POS (Jk), POS (Jk + 1)) ∈ CPPS, Ch to Jk + 1 Align.
Case 2: If the word Ch is aligned to Jk and Jk-1 is not aligned again, if (POS (Jk-1), POS (Jk)) ∈ CPPS, Ch to Jk-1 Align. FIGS. 11A and 11B illustrate a case 1 and a case 2.

For all aligned words Ch,
Case 1: Ch is aligned to Jk, and Ch + 1 is not aligned again. If (POS (Ch), POS (Ch + 1)) ∈ CPPS, align Ch + 1 to Jk. Let
Case 2: Ch is aligned to Jk, and Ch-1 is not aligned again. If (POS (Ch-1), POS (Ch)) ∈ CPPS, align Ch-1 to Jk. Let 12A and 12B exemplify case 1 and case 2. FIG.

  Next, the 1-n non-continuous type and the m-1 non-continuous type alignment unit 232 will be described. For 1-n non-continuous and m-1 non-continuous alignments, most of the words corresponding to 1 are connectives, prepositions, and particles, so these 1-n non-continuous and m -1 Discontinuous alignment is extracted in advance and converted into a dictionary. This is called a many-to-many alignment dictionary MMADic, and a sample thereof is shown in FIG. Here, one Chinese word and a plurality of non-consecutive Japanese words corresponding thereto, and conversely, one Japanese word and a plurality of non-contiguous Chinese words corresponding thereto An example of is shown.

  Here, S = CS <-> JS represents one parallel example sentence pair. Here, CS is an example sentence in the first language, and JS is a translated sentence in the second language of CS. CS = <C1, C2,..., Cm>, JS = <J1, J2,..., Jn>, Ch and Jk are words.

For all aligned words Ch, if
1) Ch is in the many-to-many alignment dictionary MMADic,
2) The translation set corresponding to Ch is a word of Jk, Js, ..., JS,
3) The translation set Jk, Js, ... is not aligned again,
If so, the word Ch is aligned with Jk, Js,.

For all aligned words Jk, if
1) Jk is in the many-to-many alignment dictionary MMADic,
2) Ch, Cs, ..., the set of translations supported by Jk are CS words,
3) The translation set Ch, Cs, ..., is not aligned again,
If so, the words Ch, Cs, ... are aligned to Jk.

  Next, the m-n continuous alignment unit 234 will be described. S = CS <-> JS represents one parallel example sentence pair. Here, CS is an example sentence in the first language, and JS is a translated sentence in the second language of CS. CS = <C1, C2,..., Cm>, JS = <J1, J2,..., Jn>, Ch and Jk are words.

  Alignment result of P using P = <Ch, Ch + 1>; MN-Alignment (P, Alignment (P)) for all aligned consecutive two words Ch and Ch + 1 Ask for. If Alignment (P) = <Jk, Jk + 1> and Jk and Jk + 1 are not yet aligned, align <Ch, Ch + 1> to <Jk, Jk + 1> .

  Alignment result of P using P = <Jk, Jk + 1>; MN-Alignment (P, Alignment (P)) for all aligned consecutive two words Jk and Jk + 1 Ask for. If Alignment (P) = <Ch, Ch + 1> and Ch and Ch + 1 are not yet aligned, align <Ch, Ch + 1> to <Jk, Jk + 1> .

  Definition: Express example sentences and their translations as CS and JS, respectively. An example sentence parallel translation pair is represented by S = CS <-> JS, and a candidate example sentence pair is represented by BS. The example sentences and their translations are expressed as character strings with ordered characters. That is, CS, JS, and S are expressed by mathematical formulas (1), (2), and (3).

  A plurality of consecutive words is represented by P. P is also expressed by a character string with an order of characters, and is expressed by Equation (4).

  Next, the definition of the common part of the example sentence translated sentence pair (group) will be described. The example sentence translation pair Sk, Sh is included in the candidate BS of the translation example sentence pair, and their common parts are expressed by equations (5) and (6).

The method for obtaining the common part of the example sentence translation pair Sk, Sh is performed by the equations (7), (8), (9), (10), and (11). Here, CWSTOP represents a prohibited character set for language 1, and JWSTOP represents a prohibited character set for language 2. For example, Japanese characters “ha”, “ga”, “wa”, etc., such as “target”, “ground”, “gain” in Chinese, etc., appear as high-frequency characters in the document.

  Next, processing when the language of P is language 1 will be described. If the common part of Sh and Sk is the following expression (12-1), it is expressed that Sh and Sh strongly support P <-> Tg. In this case, Tg is a P alignment candidate.

  If x example sentence pairs support P <-> Tg in the example sentence parallel translation candidate BS, it is defined that the support degree that Tg is an alignment candidate of P is x, and SV (P <-> Tg) = x. Then, Tg having the maximum support degree SV (P <−> Tg) is taken as the alignment result of P, and is expressed as in Expression (13-1).

  If the common part of the example sentence translation pair Sh and Sk is the formula (14-1), it is expressed that Sh and Sh weakly support P <-> Tg, and Tg is also a P alignment candidate in this case .

  On the other hand, if the language of P is language 2, if the common part of Sh and Sk is the following equation (12-2), it is expressed that Sh and Sh strongly support P <-> Tg. In this case, Tg becomes a P alignment candidate.

  If x example sentence pairs support P <-> Tg in the example sentence parallel translation candidate BS, it is defined that the support degree that Tg is an alignment candidate of P is x, and SV (P <-> Tg) = x. Then, Tg having the maximum support degree SV (P <-> Tg) is taken as the alignment result of P, and is expressed as Expression (13-2).

  If the common part of the example sentence parallel translation pair Sh and Sk is the formula (14-2), it is expressed that Sh and Sh weakly support P <-> Tg, and Tg is also a P alignment candidate in this case. .

  Next, a processing procedure of the m-n continuous alignment unit will be described. FIG. 14 is a diagram illustrating a processing flow of the m-n continuous alignment unit. The m-n continuous alignment unit 234 accesses the example sentence parallel translation dictionary 26b, and obtains a plurality of example sentence parallel translation pair candidates BS including a plurality of consecutive words P, for example (step S201). The obtained example sentence parallel translation pair candidate BS is stored in the storage unit 216. Next, a pair of common parts between the respective example sentence parallel translation pairs is extracted, and the support level of all the pair candidates is calculated (step). S202). Then, one having a high degree of support is selected as an alignment candidate (step S203).

Selection of alignment candidates is performed as follows, for example.
Here, the two alignment candidates are T1 and T2, and their support levels are x and y (where x> y). That is, SV (P <-> T1) = x and SV (P <-> T2) = y.

If x <θ1, it is determined that the alignment fails because there is no suitable candidate.
If x> = θ1 and xy> θ2, T1 is output as the alignment result.
If x> = θ1 and x / y> θ3, T1 is output as the alignment result.
However, θ1, θ2, and θ3 are non-negative real numbers.

  It is determined whether or not there is an alignment result according to the above criteria (step S204). If there is an alignment result, the result is output (step S205). When there is no alignment result here, the next processing, for example, the processing by the bipartite graph base alignment unit 220 is performed.

  FIG. 15 shows a specific example of translation from Chinese into Japanese by the m-n continuous alignment unit. A plurality of continuous words P are input, and in response to this, an example sentence including the input P is extracted from the example sentence parallel translation dictionary 26b. S1, S2,..., S5 are example sentence parallel translation pairs accompanied by the main character, and Chinese example sentences C1, C2,... C4 include a plurality of consecutive words P.

  The support level for all of the example sentence parallel translation pairs extracted in FIG. 15 is calculated. FIG. 16 shows a specific example of calculating the support level in the common part of the example sentence parallel translation pairs extracted in FIG. The support level in the common part of each example sentence parallel translation pair is determined, and as a result, the support level of the corresponding translated sentence is added.

  FIG. 17 shows a tabulation result of the support levels obtained in FIG. As a result, the degree of support for each translated sentence candidate is known.

  In FIG. 18, two translation candidates with high support are extracted from the support result tabulation result, and finally it is determined that T1 exceeds a predetermined standard, and the final alignment result is determined. Yes.

Next, returning to FIG. 7, when the alignment is desired by the many-to-many alignment unit 218, alignment is performed by the BGMWM base alignment unit 220.
Alignment here creates a weight bipartite graph using all unaligned CS word sets and JS word sets, and uses a bipartite graph-based alignment unit as shown in FIGS. Perform alignment.

  FIG. 19 shows an experiment using a test set (Open test) of 100 pairs of examples using a sports newspaper article in the region and an example sentence parallel translation dictionary in which 15,405 parallel translations are stored in the word alignment apparatus of the present embodiment. Shows the results when. F-Score is defined by F = (R + P) * 2 / (R + F). As can be seen from the experimental results, when various algorithms are executed by the automatic word alignment apparatus of this embodiment, the recall rate, accuracy, and F-Score are improved as compared with the conventional method.

  FIG. 20 is a block diagram showing a hardware configuration of the word alignment apparatus. The translation memory translation device preferably includes an input device 700, a display device 702, a main storage device 704, a storage device 706, a central processing unit (CPU) 708, and a bus 710 connecting them.

  The input device 700 includes a keyboard for inputting information by key operation, a scanner for optically reading a document or the like written on a manuscript, an input interface for inputting data from an external device, an external memory, or the like. The display device 702 includes a display that displays user input, translation results, and the like. The main storage device 704 includes a ROM or a RAM, and stores a program for controlling the operation of each unit shown in FIG. The storage device 706 includes a large-capacity storage device such as a hard disk, and stores data such as various dictionaries 26 and 124 such as an example sentence parallel translation dictionary. The central processing unit 708 controls each unit according to a program stored in the main storage device 704.

  The preferred embodiment of the present invention has been described in detail above, but the present invention is not limited to the specific embodiment, and various modifications can be made within the scope of the present invention described in the claims. Deformation / change is possible.

  For example, in the above-described embodiment, an example is shown in which a translation is generated by selecting an example sentence candidate having one non-zero permutation weight. However, the present invention is not limited to this. It may be one or more. Also, the number of dropouts may be greater than one. If the number of substitutions and omissions increases, the translation accuracy may decrease accordingly, but it can be used in a translation system with sufficient consideration.

  The translation memory translation apparatus according to the present invention is used in a machine translation system and a translation support system.

It is a figure which shows the example of whole structure of the machine translation system which concerns on an example sentence base machine translation system. It is a figure explaining the example sentence pattern base translation apparatus shown in FIG. FIGS. 3A and 3B are diagrams for explaining alignment between words included in an example sentence and a translated sentence. FIG. 3A shows a pair of input example sentences in Chinese and the translated sentence in Japanese, and FIG. Shows the ideal alignment between Japanese and Japanese words. It is the whole structure of the word alignment apparatus by bipartite graph optimal matching. It is a figure explaining bipartite graph matching. It is a figure which shows the example of the word memorize | stored in the word parallel translation dictionary, and the commentary translation of a word. It is a figure which shows the structure of the word alignment apparatus which concerns on the Example of this invention. It is a figure which shows the example of the morphological analysis of Chinese and Japanese. It is a figure which shows the operation | movement flow of the pre-processing part shown in FIG. It is a block diagram which shows the internal structure of the many-to-many alignment part shown in FIG. It is a figure which shows the example of 1-n continuous type alignment It is a figure which shows the example of m-1 continuous type alignment. It is a figure which shows the sample of many-to-many alignment dictionary MMADic. It is a figure which shows the processing flow of an m-n continuous type alignment part. It is a figure which shows the specific example translated from Chinese into Japanese by the m-n continuous type alignment part. It is a figure which shows the specific example which calculates the support degree in the common part of the example sentence parallel translation pair extracted in FIG. It is a figure which shows the total result of the support degree calculated | required in FIG. It is a figure which shows the example determined to the final alignment result from the total result of a support degree. It is a figure which shows the experimental result in the word alignment apparatus of a present Example. It is a block diagram which shows one hardware structure of a word alignment apparatus.

Explanation of symbols

10: machine translation system 12: source language text sentence input unit 14: translation memory device 16: example sentence pattern base translation device 18: word direct translation translation device 20: target language text output unit 22: automatic translation inappropriate sentence collection unit 24: learning Device 26a: Word parallel translation dictionary 26b: Example sentence parallel translation dictionary 26c: Example sentence pattern parallel translation dictionary 200: Word alignment device 202: Input unit 204: Morphological analysis unit 206: Preprocessing unit 208: Word parallel translation dictionary 210: Dictionary base alignment unit 212: China Simplified Chinese character conversion unit 214: word notation base alignment unit 216: storage unit 218: many-to-many alignment unit 220: BGM base alignment unit 222: output unit 700: input device 702: display device 704: main storage device 706: storage Device 708: Central processing unit (CPU) 710: Bus

Claims (18)

A word alignment device that performs alignment between words included in each example sentence of a second language that is a translation of the example sentence of the first language and the example sentence,
A first language word and its translation, a second language word, a commentary translation when there is no second language word translation for the first language word, and a first language word for the second language word A word bilingual dictionary that memorizes the correspondence of commentary translations when there is no word translation,
An input means for inputting an example sentence in the first language and an example sentence in the second language that is a translation of the example sentence;
Word extraction means for extracting words of the first language and the second language from the inputted example sentences of the first language and the second language, respectively;
When the correspondence relationship between the extracted first language word and the extracted second language word is included in the word parallel translation dictionary, the extracted second language word is extracted from the extracted second language word. First alignment means for aligning with words in the language;
For a word in the first language that is not aligned by the first alignment means, when the correspondence relationship between the word in the first language and the commentary translation in the second language is included in the word bilingual dictionary, the first language words from the set of all commentary translation of words to identify a substring that is not common and the other with alignment to a word of the second language as the extraction, the partial character string that particular first language Second alignment means for aligning with
A word alignment apparatus. The word alignment device further divides the word in the first language for the word in the first language that is not aligned by the second alignment means, and the correspondence between the divided word and the commentary translation in the second language is the word Third alignment means for aligning the divided words with unaligned second language words that are included in the set of all explanatory translations of the divided words when included in the bilingual dictionary; The word alignment apparatus according to claim 1. The word alignment device further includes a first language word when the first language word and the extracted second language word match for the first language word that is not aligned by the third alignment means. The word alignment apparatus according to claim 2, further comprising: a fourth alignment unit that aligns the extracted word with the extracted second language word. The word alignment apparatus according to claim 3, wherein the match of the notation is a match of Chinese characters of the first language word and the second language word. The word alignment apparatus according to claim 1, wherein the second alignment means performs alignment from one word in the first language to a plurality of consecutive words in the second language. The word alignment apparatus further stores a part-of-speech connection table indicating a connection relationship between parts of speech of two consecutive words using information in which alignment is given in advance for words in the first language and words between the second language. Storage means;
When the extracted first language word is aligned with a part of the extracted second language consecutive words and the remaining words are not aligned, the consecutive words are 6. A fifth alignment means for aligning the extracted first language word with a plurality of consecutive words in the second language when included in the part-of-speech connection table. The word alignment apparatus as described in. The fifth alignment means aligns a part of the extracted consecutive words with the extracted second language word, and the remaining plurality of words are not aligned. The word alignment apparatus according to claim 6, wherein, when a word is included in the part-of-speech connection table, the extracted continuous first language word is aligned with a second language word. The part-of-speech connection table uses a corpus to which word alignment is given, and includes the number of all two consecutive words included in the corpus and the number of two consecutive words aligned in the corpus. The word alignment apparatus according to claim 6 or 7, wherein connection of parts of speech having a certain appearance frequency is defined from a ratio. The word alignment device further includes an alignment dictionary that stores correspondence relationships between one word in the first language and a plurality of non-consecutive words in the second language that are translations thereof.
The extracted words in the first language are included in the alignment dictionary, and a plurality of non-consecutive words in the second language that are parallel translations of the words in the first language are included in the example sentences in the second language. The word alignment apparatus according to claim 1, further comprising: a sixth alignment unit that aligns the extracted words in the first language with the discontinuous words in the second language. . The word alignment device further includes an example sentence including a plurality of consecutive words in the first language extracted from the example sentence parallel translation dictionary storing the example sentence in the first language and the translation in the second language which is the translation of the example. A set of common parts between the selected example sentence parallel translation pairs, and a support level of the extracted common part for the plurality of consecutive words is calculated. The word alignment apparatus according to any one of claims 1 to 9, further comprising a seventh alignment unit that aligns the plurality of consecutive words with a plurality of words that are common parts of the second language. The support degree is calculated by selecting a common part of the corresponding translated sentence in the second language as an alignment candidate when the common part of each set has a predetermined matching degree with respect to the plurality of continuous words. And a means for counting the number of appearances of the selected alignment candidate as representing the degree of support thereof. If the number of appearances of the alignment candidate with the highest number of appearances is smaller than the first threshold, the alignment candidate with the highest number of appearances is not aligned, the number of appearances is greater than or equal to the first threshold , and the If the difference between the number of appearances and the number of appearances of the translation candidate with the next highest number of appearances exceeds the second threshold, or the number of appearances is greater than or equal to the first threshold , and the number of appearances and the next number of appearances The alignment candidate with the largest number of appearances is set as the alignment of the plurality of consecutive words in any case where the ratio of the number of appearances of the translated sentence candidates to the number of appearances exceeds a third threshold. Item 12. The word alignment device according to Item 11. The word alignment apparatus according to any one of claims 1 to 12, an example sentence bilingual dictionary storing example sentences in a first language and a second language example sentence corresponding to the first example sentence, and the example sentence parallel translation dictionary. A translation generation unit that generates a translation of a second language example sentence of the first language, and the translation generation unit generates a translation using an alignment result of the word alignment unit. A word alignment program executed by a word alignment device that performs alignment between words included in each example sentence in a second language that is a translation of the example sentence in the first language,
Inputting an example sentence in the first language and an example sentence in the second language that is a translation of the example sentence;
Extracting words of the first language and the second language from the input example sentences of the first language and the second language, respectively;
When the correspondence relationship between the extracted first language word and the extracted second language word is included in the word bilingual dictionary, the extracted first language word is converted to the extracted second language. A first alignment step that aligns to a word of
For a word in the first language that has not been aligned in the first alignment step, the word bilingual dictionary includes the correspondence relationship of the commentary translation when there is no parallel translation of the second language word for the first language word. are time, the the set of all commentary translation of words in the first language, identifying a substring that is not common and the other with alignment to a word of the second language as the extraction, the identified portion character A second aligning step to align the sequence with words in a first language;
For words in the first language that are not aligned in the second alignment step, the words in the first language are divided, and the correspondence between the divided words and the commentary translations in the second language is included in the word bilingual dictionary. A third alignment step of aligning the divided words with the unaligned second language words that are included in the set of all commentary translations of the divided words ;
A word alignment program. The word alignment program
For words in the first language that are not aligned in the third alignment step, the words in the first language are extracted when the notation of the words in the first language and the extracted words in the second language match. The word alignment program according to claim 14, further comprising a fourth alignment step for aligning words in a second language. The word alignment program further aligns the extracted first language words with a portion of the extracted second language consecutive words and the remaining words are not aligned. If the plurality of words to be included are included in the part-of-speech connection table indicating the connection relation between the parts of speech of two consecutive words prepared in advance, the extracted words in the first language are a plurality of consecutive words in the second language. The word alignment program according to claim 14, comprising a step of aligning to a word. In the word alignment program, the extracted first language word is further included in an alignment dictionary that stores correspondence between one word in the first language and a plurality of non-consecutive words in the second language, which is a translation of the word. And when the second language example sentence includes a plurality of non-consecutive words in the second language that are parallel translations of the words in the first language, the extracted words in the first language are The word alignment program according to claim 16, further comprising the step of aligning with a plurality of non-contiguous words of a language. The word alignment program
Selecting from the example sentence parallel translation dictionary an example sentence including a plurality of consecutive words in the extracted first language and its parallel translation;
Extracting a pair of common parts between the selected example sentence parallel translation pairs;
Calculating a degree of support of the extracted common part for the plurality of consecutive words;
Aligning the plurality of consecutive words with a plurality of words that are common parts of the second language based on the calculated support;
The word alignment program according to claim 16 or 17, comprising: