JPH09160918A - Translated sentence corresponding method and device therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the wide translated sentences of two languages correspond to each other with high precision. SOLUTION: An input part 110 inputs the correspondence text of two languages such as Japanese and English from a storage device 10, etc. A morpheme analyzing part 120 performs a morpheme analysis for each inputted text. A degree of similarity calculating part 130 calculates the degree of similarity of the word of the both languages as the mutual information capacity in the text from the morpheme analysis result and further selects a word pair with high reliability in a statistical qualification. A sentence correspondence estimating part 140 narrows down the sentence correspondence possible relation by using the degree of similarity and an existing bilingual dictionary. A post- processing part 150 selects a sentence correspondence pair having a prescribed number of times of support for the narrowed down sentence correspondence possible relation. An output part 160 outputs this selected sentence correspondence pair to a storage device 20, etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a bilingual sentence associating method and apparatus, and more particularly, to a bilingual sentence used for a natural language system such as a machine translation or a knowledge base system to automatically learn knowledge from a bilingual text. The present invention relates to an associating method and device.

[0002]

2. Description of the Related Art Conventional bilingual sentence mapping is performed mainly between languages such as English and French that have very similar structures and vocabularies, and these are information such as the number of words and the number of characters included in a sentence. It was common to associate parallel translations with. On the other hand, regarding bilingual translations such as Japanese and English, there are a method of using only a bilingual dictionary and a method of using statistics as post-processing after using a bilingual dictionary using a dynamic programming method.

[0003]

As described above, the conventional bilingual sentence associating method has dealt with texts having a similar structure and relatively easy to associate. However, between languages such as Japanese and English, which have completely different structures and thinking methods, it is common for the translated texts to be translated in a different way and to have their contents deleted. . In such cases, it is important to properly combine the statistical information from the data with the existing knowledge source dictionary. The advantages and disadvantages of statistical information and dictionary information are summarized as follows.

Advantages of statistical information: Since data-dependent information can be obtained, it is possible to find a translation relation suitable for the context of the text. Further, in a language such as Japanese that requires word segmentation (morphological analysis), it is an advantage that information can be retrieved even if the word segmentation is incorrect. Disadvantages of statistical information: In order to obtain reliable statistical information, the target word must appear multiple times in the data. Considering that many words appear only once or twice, the number of words for which statistical information can be obtained is limited. Advantages of dictionary information: Information can be obtained even for words that appear only once. Disadvantages of dictionary information: There are various possible translations for one word, and the ones used in the data are not always listed in the bilingual dictionary. Moreover, if there is an error in the stage of morphological analysis, correct dictionary lookup is impossible. As can be seen from these, the advantages and disadvantages of statistical information and dictionary information are in a complementary relationship.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the conventional problems and provide a highly accurate parallel translation sentence associating method and apparatus in which statistical information and dictionary information are properly combined.

[0006]

According to the present invention, when corresponding texts in two languages are given, the similarity calculation means
The degree of similarity between words in both languages is calculated as the amount of mutual information in the data, and only those with a high degree of reliability are selected by a statistical test such as t-test. Next, the sentence correspondence estimation means narrows down the range of possible sentences using this similarity and the information of the existing bilingual dictionary. Using this refined information,
Further, the above operation is repeated in the similarity calculation means and the sentence correspondence estimation means. By repeating this operation, the set of available sentences is gradually narrowed down, and the desired sentence association is finally obtained.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, as one embodiment of the present invention,
The case where corresponding texts in Japanese and English are given is explained.

FIG. 1 shows a system configuration diagram of a bilingual sentence associating device according to an embodiment of the present invention. Book translation matching device 1
00 is an input unit 110, a morphological analysis unit 120, a similarity calculation unit 130, a sentence correspondence estimation unit 140, a post-processing unit 150, an output unit 160, a storage unit 170 used as a work area for each of these units, and an existing unit. It consists of a bilingual dictionary 180. Reference numeral 10 is a storage device in which Japanese and English and corresponding text data are stored, and 20 is a storage device in which associated bilingual sentence pairs are stored. The input means for the corresponding text data does not necessarily have to be the storage device.

The input unit 110 inputs corresponding texts in two languages, Japanese and English, from the storage device 10 and stores them in a predetermined work area of the storage unit 170. The morphological analysis unit 120 stores the corresponding texts in Japanese and English in the storage unit 17.
0 is extracted from a predetermined work area, morphological analysis is performed on each, and the result is stored in a predetermined work area of the storage unit 170. The similarity calculation unit 130 includes a storage unit 170.
From the morphological analysis results in the given work area, the correspondence relationship between words in both languages is calculated, and the mutual information amount is calculated.
Further, a highly reliable word pair is selected by a statistical test (t-test) and stored in a predetermined work area of the storage unit 170. The sentence correspondence estimation unit 140 counts the number of times the correspondence between the Japanese sentence i and the English sentence j is supported for a word pair in a predetermined work area of the storage unit 170, using a pre-prepared bilingual dictionary 180. Sentence correspondence relationships are narrowed down by a threshold value and stored in a predetermined work area of the storage unit 170. The post-processing unit 150 selects a sentence correspondence pair having a predetermined number of support times from the sentence correspondence correspondence in the predetermined work area of the storage unit 170 and stores it in the predetermined work area of the storage unit 170. The output unit 160 outputs to the storage device 20 the sentence-corresponding pair in the predetermined work area of the storage unit 170 selected by the post-processing unit 150.

FIG. 2 shows the similarity calculator 13 in FIG.
0 shows the connection relationship between the sentence correspondence estimation unit 140 and the post-processing unit 150. Here, the similarity calculation unit 130 and the sentence correspondence estimation unit 14
0 forms a loop via the work area of the storage unit 170, and the sentence correspondence range is narrowed down by repeating the processes of both.

FIG. 3 shows a series of processing steps in this embodiment. First, the morphological analysis unit 120 performs morphological analysis on both the corresponding Japanese text and English text, and selects only the necessary part-of-speech word (step 300). In the subsequent correspondence, only the words extracted here are used. An initial sentence correspondence relationship is created from the number of sentences in the input Japanese-English text. In this initial relationship, the beginning and end of each text correspond to each other, and other correspondences have a width. The width of the correspondence is small at both ends of the text,
The closer to the center of the text, the larger.

FIG. 4 shows an example of this correspondence possibility. The width of the correspondence is small at both ends of the text, and increases toward the center of the text. There are as many correspondence relationships as there are Japanese sentences.

Next, the similarity calculator 130 estimates word correspondences from the correspondence relationships (step 310).
The operation of the similarity calculation unit 130 will be described below.

It is now assumed that the j-th Japanese word in the corresponding pair i is J _ij and the k-th English word in the corresponding pair i is E _ik . Also, _let N (J _ij ) be the number of corresponding pairs in which the word J _ij appears. However, one occurrence word is managed so as not to be counted twice in a plurality of pairs. At this time, the day of the corresponding pair i,
The similarity between English words J _ij and E _ik is the mutual information I
(J _ij , E _ik ). Here, Pr is the probability and n is the total number of pairs (that is, the number of sentences in Japanese text).

[0015]

[Equation 1]

This mutual information indicates the rate of appearance of word pairs J _ij and E _ik , and by using this, the closeness of Japanese and English words can be measured. However, the mutual information can be large even for infrequent words, but this is statistically unreliable. Therefore, the similarity calculation unit 130
Then, a statistical test (t-test) is also performed, and only the reliable word pairs are extracted. The operation of obtaining the mutual information amount is performed for all word combinations (having a necessary part of speech) included in the correspondence relationship.

Next, the sentence correspondence estimation unit 140 narrows down the sentence correspondence correspondence relationship using the word pairs obtained by the similarity calculation unit 130 and the existing bilingual dictionary (step 320).

The sentence correspondence estimation unit 320 counts the number of times the correspondence between the Japanese sentence i and the English sentence j is supported in the following steps. st and dic are parameters given externally, and are points added when supported by statistics and bilingual dictionaries, respectively. Usually, st is set larger than dic. Step 1: This operation is applied in the order of word pairs with the largest mutual information. That is, if the Japanese sentence i and the English sentence j include the pair and there is no other English sentence included in the corresponding pair of the Japanese sentence i including the English word, the combination of the Japanese sentence i and the English sentence j is st. Add. It is also possible to add a non-tolerance condition to this step. Step 2: If the Japanese sentence i and the English sentence j include a word pair in the bilingual dictionary, and there is no other English sentence included in the corresponding pair of the Japanese sentence i including the English word, the Japanese sentence i and the English sentence j.
Add dic to the combination. Step 3: Correspondences exceeding the thresholds of Steps 1 and 2 are confirmed as reliable correspondences (this correspondence is called an anchor). Construct a new sentence-capable relationship from the anchor column as input to the next iteration. The correspondence between the portions sandwiched by the two anchors has a width, but the width is smaller as it is closer to the anchor and larger as it is closer to the center of the anchor.

The above-described processing of the similarity calculation unit 130 and the sentence correspondence estimation unit 140 is repeated until the sentence correspondence relation converges (step 330), thereby obtaining the sentence correspondence relation between the input Japanese-English texts. Can be done.

This method has the following advantages over the existing method that uses statistics as post-processing after using an existing bilingual dictionary based on dynamic programming. (1) In the existing method, since the correspondence is first made with the dictionary, the accuracy rate is remarkably lowered in the texts in which the vocabulary is not posted in the dictionary, such as texts in specialized fields. Since the statistical processing of the existing method is performed based on the result of the association by the dictionary in the first step, it is impossible to obtain a correct result when the correct answer rate in the first step is low. In addition, the existing method has a problem that it is greatly affected by the accuracy rate of the morphological analysis unit. The method of the present invention solves these problems. (2) Japanese and English texts often include parts that do not correspond to each other. In addition, the correspondence between Japanese sentences and English sentences is often crossed (when Japanese sentences i correspond to English sentences j, Japanese sentences with numbers smaller than i correspond to English sentences larger than j or vice versa). . In the existing method, such a problem cannot be dealt with because local mapping is performed by dynmic programming. On the other hand, the method of the present invention in which the anchor is set while viewing the entire text can address the above problem.

Next, the post-processing section 150 derives a final result from the converged sentence correspondence correspondence relationship (step 340). In the sentence correspondence relationship, Japanese sentences with low support frequency have many corresponding English sentences. Therefore, in the post-processing unit 150, when there is a significant difference in the number of support times between the correspondences, only a sentence having a large number of support times is selected as a correspondence relationship. It is determined that there is no corresponding English sentence.

Although the correspondence between Japanese and English sentences has been described above as an embodiment of the present invention, it goes without saying that the present invention is not limited to this.

[0023]

As described above, according to the present invention,
It is possible to automatically and highly accurately associate sentences included in texts of a wide range of two countries. Therefore, the corpus with correspondence obtained here is used in a system such as a machine translation or an example sentence search system, and can also be used as an input of a system for automatically learning knowledge.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram as an embodiment of the present invention.

FIG. 2 is a diagram showing a connection relationship of main parts in FIG.

FIG. 3 is a flowchart illustrating the operation of the embodiment of the present invention.

FIG. 4 is a diagram illustrating a sentence correspondence pair.

[Explanation of symbols]

 100 bilingual association device 110 input unit 120 morphological analysis unit 130 similarity calculation unit 140 sentence correspondence estimation unit 150 post-processing unit 160 output unit 170 storage unit 180 bilingual dictionary

Claims (2)

[Claims] 1. A method for automatically associating sentences included in bilingual texts, wherein bilingual texts are provided based on statistics when bilingual texts are given. Repeat the process of calculating the degree of similarity between words and the process of estimating the sentence correspondence using the similarity and the existing bilingual dictionary, gradually narrowing down the set of sentences that can be dealt with, and finally obtaining the desired sentence correspondence. A method for associating bilingual texts, which is characterized by obtaining an index. 2. An input means for inputting corresponding texts in two languages, a morpheme analysis means for performing a morpheme analysis on each input text, a correspondence possibility relationship is calculated from a morpheme analysis result, and a mutual information amount is obtained. The number of times the correspondence between a sentence in one language and a sentence in the other language is supported using a similarity calculation means that selects a word pair by a statistical test and a prepared bilingual dictionary for the selected word pair. Count
Sentence correspondence estimating means for narrowing down the sentence correspondence possibility with a predetermined threshold, post-processing means for selecting a sentence correspondence pair having a predetermined support frequency for the narrowed sentence correspondence relation, and selected sentence correspondence A parallel translation sentence associating device, comprising: an output unit that outputs a pair as a final result.