KR20030056655A - Similar sentence retrieval method for translation aid - Google Patents

Abstract

PURPOSE: A method for searching a similar sentence on a translation support system is provided to improve accuracy of the system by offering a similarity measure between sentences for searching the most similar example from a translation memory, and outputting a searched source sentence and the corresponding translation sentence. CONSTITUTION: The translation memory(105) comprises the source sentence, a morpheme analysis result of the source sentence, and the translation sentence by performing the morpheme analysis of a parallel corpus(101) in not processed form through a morpheme analyzer(102). An index reverse-file(104) is formed by extracting an index word from each source sentence forming the translation memory(105). A filtering part(106) separately extracts the morpheme corresponding to a noun, a verb, and an adjective used as the index word from the inputted source sentence. A searching part(107) loads a searching result after searching the candidate sentences provided from the filtering part(106) through the translation memory(105). A similarity calculation part(108) calculates the similarity applying a weight of each part for the candidate sentences by using an edit distance method.

Description

How to search for similar sentences in translation support system {SIMILAR SENTENCE RETRIEVAL METHOD FOR TRANSLATION AID}

The present invention relates to a technique for retrieving a similar sentence in a translation support system, and more particularly, to a structure of a translation memory composed of a pair of original text and a translation sentence and a grammatical / structural structure among original texts of a translation memory for a translation support system. A method for searching similar sentences in a translation support system suitable for searching for the most similar sentences.

In the general machine translation system, natural translation quality cannot be obtained at this time. The reason is that the primitive language analysis technique is not yet perfect, and the conversion technique for converting the primitive language into the target language is also insufficient at present.

On the contrary, the translation support system does not provide a fully automatic translation function. However, since the translation support system searches for the most similar sentence and the corresponding band sentence in the translation memory, it helps the user to translate. It is much more practical than machine translation system.

However, since most translation support systems use only string matching techniques for words appearing in the input text, they still have a disadvantage in that they can only output sentences that are superficially matched.

Therefore, in order to solve the above problems and to realize a higher quality result in consideration of the structural and grammatical components of the sentence, not only surface word-based matching but also different weights according to morphological analysis results and parts of speech. A technical solution for extracting similar sentences is desired.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described needs, and provides a measure of similarity between sentences for retrieving example sentences most similar in grammatically and structurally from a user-proposed input sentence using a translation memory as a resource. It is an object of the present invention to provide a method for retrieving similar sentences in a translation support system that realizes more accurate system performance by outputting original sentences and corresponding translation sentences retrieved from the translation memory by this scale.

In order to achieve the above object, the present invention comprises a first step of constructing a translation memory from a parallel corpus composed of a pair of original text and band text, and constructing an inverse file of index words for quickly accessing the translation memory; A second step of first filtering candidate sentences before calculating similarity by applying the information of the index inverse file and the matching weights of nouns, verbs, and adjectives; And calculating each similarity between the input original text and the candidate sentences according to the similarity calculation method defined by loading the filtered candidate sentences from the translation memory, and outputting the original text and the corresponding band text in order of similarity. A similar sentence search method in a translation support system is provided.

1 is a block diagram of a translation support system applied to a method according to the present invention;

2 is a diagram illustrating a similar sentence search method according to an embodiment of the present invention.

3 is a diagram for describing a similarity calculation process according to a similar sentence search method according to another embodiment of the present invention.

<Description of the code | symbol about the principal part of drawing>

101: parallel corpus 102: stemmer

103: input unit 104: index inverse file

105: translation memory 106: filtering unit

107: search unit 108: similarity calculation unit

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

Prior to the description, terms used in the present invention are defined as follows.

First, the translation memory is composed of the original text, the result of the morphological analysis of the original text, and the band text. The larger the size of the translation memory, the higher the probability of finding a sentence similar to the input text.

The index inverse file is an index word extracted from the result of morphological analysis of all sentences constituting the translation memory, and the corresponding word is a key, and includes the sentence number and the position information in the sentence. .

The similarity between sentences is calculated according to the defined similarity and represents a measure of grammatical and semantic similarity between two sentences.

In the following embodiment, a translation support system using a translation memory will be described as an example. However, it is noted that the present embodiment is not limited to the translation support apparatus using the translation memory.

1 is a block diagram of a translation support system applied to a method according to the present invention.

First, the translation memory 105 is a morphological analysis of the raw corpus parallel corpus 101 through the morphological analyzer 102 is composed of a set of three elements, the original text, stemming results of the original text, band text, In this process, an index word is extracted from each original text constituting the translation memory 105 to form a separate index inverse file 104. The index inverted file 104 is information about a noun, a verb, and an adjective used as an index word among the stemming results of all original texts constituting the translation memory 105, that is, the information of the morpheme of the number of sentences. , Sentence number and morphological position information.

The text input through the input unit 103 is analyzed for each word constituting the original text through the morpheme analyzer 102, and at the same time, the parts of speech of the words are determined.

The filtering unit 106 separately extracts only morphemes corresponding to nouns, verbs, and adjectives used as index words from the input original text. The candidate sentences including the index vocabularies of the input original text are virtually constructed by referring to the information of the index inverse file 104 based on the extracted index words of the input original text. The constructed virtual candidate sentences are composed of only index words, not sentences stored in the actual translation memory 105. By applying the nouns, verbs, and adjective weights to the virtual candidate sentences constructed as described above, the matching rate between the input original text and the virtual candidate sentences is first filtered.

The reason for such filtering is to reduce the overhead of the similarity calculation by extracting only some candidate sentences before calculating the similarity between all the texts constituting the translation memory 105 and the input text.

The search unit 107 searches for the candidate sentences provided by the filtering unit 106 through the translation memory and loads a search result.

The similarity calculator 108 calculates the similarity using a "Edit Distance" technique to which the parts-of-speech weights are applied to the candidate texts of the translation memory 105.

Lastly, the most similar sentences of the translation memory 105 are ranked according to the similarity calculated by the similarity calculator 108, and then the printing apparatus 111 or the display apparatus through the printing unit 109 or the display control unit 110. Output to (112).

Hereinafter, with reference to the above-described configuration, with reference to Figures 2 and 3 attached to the similar sentence search implementation process according to a preferred embodiment of the present invention will be described in more detail.

First, FIG. 2 is a diagram illustrating a filtering process as a similar sentence search method according to an exemplary embodiment of the present invention.

As shown in FIG. 2, the index inverse file content 201 is morphologically analyzed for each text constituting the translation memory 105 as described above, and extracts only nouns, verbs, and adjectives used as index words. It includes a sentence number of the word and position information in the sentence.

When the input text 202 is entered as the first step of the filtering unit 106, the parts of each word of the input text are determined through the morphological analysis of the morpheme analyzer 102, and the nouns, verbs, and adjectives used as double index words. Extract only the word corresponding to This appears in the index word 203 extracted from the input text 202 of FIG.

Next, using the sentence number information and the morphological position information with reference to the index inverse file 104 for the extracted index word of the input original text, a reconstructed candidate sentence 204 including the index word is generated.

The matching weights between the input original text and the candidate sentences are obtained by applying matching weights of nouns, verbs, and adjectives to the candidate sentences 204 configured as described above. This matching weight is shown at 205 of FIG.

In this case, the matching weight used may be expressed as Equation 1 below.

Matching weight = (number of matched nouns × noun weight) + (matched verbs × verb weights)

Here, a term refers to a verb and an adjective.

Only the sentences in the translation memory that are likely to be candidates can be extracted using the obtained matching weights.

The filtering process as described above may significantly reduce the burden of calculating the similarity between sentences for all sentences in the translation memory 105 described later.

3 is a diagram illustrating a similarity sentence searching method according to another embodiment of the present invention.

In this process, the actual candidate sentences are loaded from the translation memory 105 using the results of the filtering unit 106 described above, and the similarity between the sentences between the candidate sentences and the input original text is calculated.

Prior to the description of FIG. 3, "Edit Distance" applied to the present embodiment quantitatively indicates a difference between two sentences. That is, "Edit Distance" means that when the sentence A is changed to the sentence B, the number of words to be deleted and the number of inserted words are summed based on the sentence A. However, if the "Edit Distance" is used as it is in the translation support tool, it only considers whether the surface representation is matched, and the result is quite different from what the user wants. Therefore, the present invention is characterized by applying the "Edit Distance" considering the following matters.

First, considering the parts of speech of words as well as surface words, different matching weights are given according to the parts of speech of the matched words. The reason for this is to consider the structural similarity of sentences. In other words, the candidate sentences in the translation memory are different from the surface words, but structurally similar candidate sentences can be presented as a result. For example, if the matching weights of verbs, dictionaries, and endings are greater than nouns, sentences that are structurally similar to the input text are likely to result.

Second, the matching operation, the insert operation, and the delete operation are used as basic operations of "Edit Distance". For this reason, it can be said that the insert operation and the delete operation perform a kind of normalization role for the case where the candidate sentence is long or short. That is, even though the number of words to which the long candidate sentence A is short and the short candidate sentence B is identical, the candidate sentence B may be determined to be similar to the candidate sentence A by the insertion and deletion operations.

In summary, the present embodiment is characterized by using a weighted "Edit Distance" technique.

Table 1 below shows the types of Edit operations used in the present invention and their weights.

Type of Edit Operation weight Same -10 insertion 2 delete One

Table 2 below shows weights according to parts of speech of matching words used in the present invention.

Parts of speech weight verb 10 Screening 8 Noun central 5 Etc 3

The operation-specific weights and the parts-of-speech matching weights shown in Tables 1 and 2 described above are heuristically determined.

The above-described matters are related to the "Edit Distance" to which the parts of speech are weighted, and the following are factors to be considered when calculating the similarity between sentences.

First, consider the distribution of words that appear in the input text. This is because, for example, an input text includes a compound noun, a candidate sentence containing the compound noun in the same form, and a candidate sentence containing the compound noun in a decomposed form exist. In this case, when both candidate sentences have the same number of matching words but the word distribution of the input source text is considered, candidate sentences including the corresponding compound nouns in the same form may be determined to be more similar.

The similarity between sentences used in the present invention may be expressed as in Equation 2 below.

Similarity between sentences =

(For each matched word) (part of speech weight by matched word × 'match' operation)

+ ('Insert' operation weight × number of 'insert' operations performed)

+ ('Delete' operation weight × 'Delete' operation count)

The similarity result between sentences obtained by applying the above Equation 2 can be said that the lower the similarity from the viewpoint of 'Cost'.

As shown in FIG. 3, when the stemming analysis result 301 of the input original text 202 and the stemming analysis result 302 of one candidate sentence 1 in the translation memory 105 are given, Equation 2 described above is given. When calculating the similarity between the input text 202 and the candidate sentence 1 (302) by applying, it can be seen that the result value of '-329'.

Meanwhile, in the morpheme analysis result 302 of the candidate sentence 1, the part shown in bold with the underline indicates a part matching the word of the input original text.

As mentioned above, although this invention was demonstrated concretely based on the Example, this invention is not limited to this Example, A various deformation | transformation, for example, is implemented as a program and can be read by a computer within the range which does not deviate from the summary. It can of course be stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in the form.

As described above, according to the present invention, by performing filtering and similarity calculation between sentences, the translation support system searches for and presents the translation example most similar to the original text to be translated by the user, thereby improving the performance of the translation support system. It is effective

Claims (4)

In the translation support system to search for sentences similar to the original text entered by the user through the translation memory, A first step of constructing an inverse file of the translation memory and the index word from the parallel corpus; A second step of filtering candidate sentences having high similarity by comparing input sentences presented by a user with sentences of the configured translation memory; And calculating a similarity with the actual input sentence with respect to the filtered candidate sentences, and outputting the original sentence and the band sentence in the order of the similar sentences. The method of claim 1, The first step is, Performing a morphological analysis on each of the original texts and band texts constituting the parallel corpus, and forming a translation memory in the form of original texts, stemming results of original texts, and band texts; Extract the words belonging to nouns and verbs used as index words among the morphologically analyzed original texts, and use the words as a key, the sentence number where the word appears, and the morpheme position information in the sentence. And constructing an index inverse file. The method of claim 1, The second step, Extracting only nouns and verbs used as index words among words constituting the input original text; Extracting a sentence number including the corresponding words from the index inverse file with respect to the index words of the input original text extracted in the step; Virtually constructing sentences including only index words of the input original text as a result of the sentence number extracting step, and extracting only possible candidate sentence numbers by applying matching weights of nouns and verbs to the virtual sentences; Similar sentence search method in a translation support system, characterized in that. The method of claim 1, The third step, Calculating similarity between sentences using a weighted " Edit Distance " method and a word distribution.