KR20240074344A - Measurement of Chinese Language Translation Quality of Automatic Translator Based on Representative Language - Google Patents

Abstract

The present invention includes a representative word database creation step to create representative words for words with the same meaning among words, and an archaic Chinese language parallel corpus that includes an archaic Chinese language text and a translation of the original Chinese language text to be used as the correct answer. A step of generating an archaic Chinese language parallel corpus, an automatic translation generation step of generating an automatic translation of the original text of an archaic Chinese language from the archaic Chinese language parallel corpus generation step using an automatic translator, and a representative word database creation step. A normalized translation generation step of generating a normalized translation, which is an automatic translation for evaluating translation rates, by replacing words applied to the automatically translated text created in the automatic translation text generation step with representative words using representative words stored in the language database, and the representative word database Using the representative words stored in the representative word database in the creation stage, words applied to the translated sentences included in the archaic Chinese parallel corpus in the generation stage of the archaic Chinese language parallel corpus are replaced with representative words to create a normalized evaluation set that is the correct answer for evaluating the translation rate. a normalized evaluation set generation step, and a translation evaluation step of evaluating the accuracy of the translation using N-gram using the normalized translation from the normalized translation generation step and the normalized evaluation set from the normalized evaluation set generation step. It consists of creating representative words for words with the same meaning among words, going through a process of replacing representative words in the correct answer and automatic translation, and evaluating them to reflect a higher degree of similarity even if the external form is different. We provide a method for measuring the translation quality of archaic Chinese characters by an automatic translator based on representative words that can accurately evaluate accuracy.

Description

{Measurement of Chinese Language Translation Quality of Automatic Translator Based on Representative Language}

The present invention applies representative words for words with the same meaning among words and applies them when evaluating the translation rate of automatic translation, thereby improving the accuracy of the similarity of translation. It is about quality measurement methods.

Currently, the BLEU measurement method is mainly used as a method to automatically measure the translation quality of an automatic translator (or automatic translation engine).

The BLEU measurement method creates a set of correct answers by using multiple translated texts as the correct answers to the original translated text. Afterwards, through NGRAM matching between the set of correct answers and the automatic translation results, we quantitatively measure how similar the automatic translation results are to the correct translation.

The conventional method of automatically evaluating translation quality used for experiments in laboratories is to evaluate the translation through indicators such as BLEU (bilingual evaluation understudy) and METEOR (An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments), and evaluate the original text. The goal is to analyze the degree of matching between the and target references. For example, BLEU focuses on the degree of matching of continuous strings, that is, n-grams. Indicators such as METEOR can be supplemented based on BLEU to not only match strings with the same text, but also identify strings with different texts but the same meaning by introducing some additional resources.

However, in the field of oral translation, the purpose of interpretation is not to interpret the speaker's content without leaving out a single word. Sometimes, to reduce time delays and adapt to local culture, interpreters typically delete some of the speaker's content and shorten sentences that do not contain much information, thereby conveying key information to the audience in as much real time as possible. In such a situation, if a conventional automatic evaluation method is used, the interpretation text is likely to receive a relatively low score due to excessive missing information, resulting in a situation in which the evaluation result is inaccurate. For example, when evaluated using conventional methods, the interpretation rate of interpreters with many years of experience is only about 70%, and the acceptable level of interpreted sentences is less than 85%.

Conventional translation quality evaluation methods have high labor costs, and human evaluation is limited by the translation level of different evaluators, so there is a large variation in evaluation results. A separate reference translation must be constructed for each interpretation situation, and at the same time, conventional methods cannot distinguish important information in the interpretation text, so the quality of the interpretation text can only be measured based on fidelity.

Due to these problems, Republic of Korea Patent Publication No. 10-2015-0029931 (hereinafter referred to as 'Patent Document 1') was proposed.

Patent Document 1 includes a storage unit that stores a plurality of answer sets, an extended dictionary, a language model, and an extended answer set, and one or more correct answers to the translated original text based on the plurality of answer sets and the extended dictionary stored in the storage unit. A reference extension unit that generates a statement and a set of extended correct answers are generated using the correct answers that have successfully been authenticated based on whether the user terminal approves the correct answers, and the set of extended correct answers is stored in the storage unit. It consists of a reference expansion authentication unit, so that the translation quality can be improved by expanding the correct answers in the correct answer set.

(Patent Document 1) KR 10-2015-0029931 A Apparatus and method for expanding the set of correct answers for measuring translation quality of an automatic translator

As described above, Patent Document 1 consists of expanding the correct answer of a set of multiple correct answers, but this is simply using a dictionary and multiple language models, so words with the same meaning but different external characteristics still have different meanings. There was a problem with the translation being interpreted and evaluated differently.

To solve the above problems, the method of measuring the translation quality of archaic Chinese characters by an automatic translator based on representative words according to the present invention creates representative words for words with the same meaning among words and represents them in the correct answer and the automatic translation. We provide a method for measuring the translation quality of archaic Chinese characters by an automatic translator based on representative words that can accurately evaluate the accuracy of translation by reflecting higher similarity even if the external form is different by evaluating them after going through a word substitution process. It has a purpose.

The present invention creates representative words for words with the same meaning among words, goes through a representative word substitution process in the correct answer and an automatic translation, and then evaluates them to reflect a higher degree of similarity even if the external form is different, thereby improving the accuracy of translation. It is a useful invention that allows accurate evaluation of

Figure 1 is a block diagram showing a method of measuring the translation quality of archaic Chinese characters by an automatic translator based on representative words according to the present invention.
Figure 2 is a block diagram showing the normalization process of translated text in the present invention.

Hereinafter, the present invention will be examined in more detail using the attached drawings.

As shown in Figure 1, the present invention includes a representative word database creation step of creating representative words for words with the same meaning among words, and a translation of the original Chinese archetypal text to be used as the original text and the correct text. An archaic Chinese language parallel corpus generation step of generating an archaic Chinese language parallel corpus, an automatic translation generation step of generating an automatically translated text obtained by translating the archaic Chinese text from the archaic Chinese parallel corpus generation step using an automatic translator, and the representative word. A normalized translation generation step of generating a normalized translation, which is an automatic translation for evaluating the translation rate, by replacing the words applied to the automatic translation created in the automatic translation generation step with representative words using data stored in the database, and the representative word stored in the representative word database. A normalized evaluation set generation step of using data to replace words applied to the translations included in the archaic Chinese parallel corpus in the archaic Chinese parallel corpus with representative words to generate a normalized evaluation set that is the correct answer for evaluating the translation rate; It consists of a translation evaluation stage in which the accuracy of the translation is evaluated using N-gram using the normalized translation in the normalized translation generation stage and the normalized evaluation set in the normalized evaluation set generation stage.

First, the representative word database creation stage is data used to evaluate translation quality by creating representative words for words with the same meaning, representing the meaning of synonyms, synonyms, dialects, archaic words, slang words, and foreign words among words. This is a step to reduce errors that occur when evaluating words based on external characteristics by selecting words that can be used.

Next, the step of creating an archaic Chinese parallel corpus is the step of creating an archaic Chinese parallel corpus that includes the archaic Chinese text to be translated and the translation of the archaic classical text by an expert.

Next, the automatically translated text generation step is the step of generating an automatically translated text through an automatic translator to evaluate the original Chinese text in the archaic Chinese language parallel corpus generation stage.

Here, in the step of creating the archaic Chinese language parallel corpus, the original text and translation of the archaic Chinese language included in the archaic Chinese parallel corpus may include a number of sentences, and the automatic translation in the automatic translation generation step automatically only includes all or part of the archaic Chinese language parallel corpus. It may be translated material.

Next, the normalized translation generation step uses the representative words created in the representative word database generation step to replace the words applied to the automatic translation created in the automatic translation generation step with representative words to create a normalized automatic translation for evaluating the translation rate. This is the step of creating a translation.

Next, the normalized evaluation set generation step uses the representative words created in the representative word database creation step to replace the words applied to the translated text included in the ancient Chinese parallel corpus in the archaic Chinese language parallel corpus generation stage with representative words, thereby increasing the translation rate. This is the step of generating a normalized translation, which is the correct answer for evaluation.

In other words, in the present invention, the normalized translation generation step and the normalized evaluation set generation step substitute words for words that have the same meaning but different external forms into representative words, so that words are given the same meaning even though their external forms are different. By evaluating the similarity between the two, the reliability of the translation rate evaluation can be improved.

Here, the normalization process of the normalized translation in the normalized translation generation step and the normalization process of the normalized evaluation set in the normalized evaluation set generation step include a morpheme analysis step of analyzing the morphemes of the input Korean sentence, and after the morpheme analysis step. A representative word search step of searching for representative words according to morphemes in the representative word database, a representative word substitution step of substituting representative words searched in the representative word search step into each translation, and sentences normalized through the representative word substitution step. It can be accomplished through a normalized sentence completion step that completes .

Next, the translation evaluation step is a step in which the accuracy of the translation is evaluated using N-gram using the normalized translation from the normalized translation generation step and the normalized evaluation set from the normalized evaluation set generation step.

The effects of the present invention will be examined through an example of the method for measuring the translation quality of archaic Chinese characters in an automatic translator based on representative words according to the present invention, which consists of the steps described above.

First, in order to properly evaluate the accuracy of the automatic translator, the present invention applies representative words for words with the same meaning to the correct translation and the automatic translation and then compares them.

First, let's look at an example of a normalized evaluation set:

Representative words for some of the data created in the database creation stage for representative words are defined as in [Table 1].

With the representative words defined as above, a normalized evaluation set can be created by applying the representative words in [Table 1] to the human translation of the ancient Chinese text as shown in [Table 2] below.

Here, the process of generating the normalized evaluation set is performed through the same process as in FIG. 2.

Meanwhile, looking at an example of the normalized translation generation step,

Representative words for some of the data created in the database creation stage for representative words are defined as in [Table 3].

With the representative words defined as above, a normalized translation can be created by applying the representative words in [Table 3] to the automatic translation of the ancient Chinese text through an automatic translator as shown in [Table 4] below. there is.

Here, the process of generating the normalized translation is carried out through the same process as in FIG. 2.

Normally, in the case of words with the same meaning, people recognize them even if their external characteristics are different, but in terminals for evaluation other than humans, when evaluating the translation rate, they may recognize them as different words and not recognize them as correct translations.

However, as shown in [Table 1] and [Table 3], the present invention creates representative words for words that have the same meaning but different external characteristics, replaces the words included in each translation with representative words, and then replaces them with representative words. By evaluating the translation accuracy of normalized sentences using an n-gram similarity comparison method, the accuracy of automatic evaluation can be increased.

Here, in n-gram similarity, “n-gram” means a list of n consecutive words, which means that there are n tokens.

In similarity comparison, the similarity is evaluated based on the number of identical tokens in the two sentences relative to the total number of tokens created by tokenizing the sentences into bi-grams, tri-grams, etc. In the present invention, sentences normalized to representative words are used. By greatly expanding the number of meaningful identical tokens by comparing n-grams, it has the effect of reflecting semantic similarity compared to vocabulary-based comparison, thereby improving the accuracy of automatic evaluation.

Although the above-described embodiment describes a preferred embodiment of the present invention, the present invention is not limited thereto and may be modified and implemented in various forms without departing from the technical spirit of the present invention. It is stated that this is obvious to those skilled in the art.

Claims (3)

A representative word database creation step of creating representative words for words with the same meaning among words;
An archaic Chinese language parallel corpus generation step of generating an archaic Chinese language parallel corpus that includes the original text of the ancient Chinese language and the translation of the original text to be used as the correct answer;
An automatic translation generation step of generating an automatic translation of the original text from the ancient Chinese language parallel corpus generation step using an automatic translator;
Normalization to generate a normalized translation, which is an automatic translation for evaluating translation rates, by replacing the words applied to the automatically translated text created in the automatic translation text generation step with representative words using the representative words stored in the representative word database created in the representative word database creation step. Translated text generation step;
Using the representative words stored in the representative word database in the representative word database creation step, the words applied to the translated text included in the archaic Chinese parallel corpus in the archaic Chinese language parallel corpus in the archaic Chinese language parallel corpus are replaced with representative words to normalize the correct answer text for evaluating the translation rate. A normalized evaluation set generation step of generating an evaluation set;
A representative word characterized in that it includes a translation evaluation step of evaluating the accuracy of the translation using N-gram using the normalized translation in the normalized translation generation step and the normalized evaluation set in the normalized evaluation set generation step. A method of measuring the translation quality of ancient Chinese characters by an automatic translator based on .
According to claim 1, in the step of creating the representative word database, the representative word database is characterized by selecting words that can represent the meaning of synonyms, synonyms, dialects, archaic words, slang words, and foreign words among words. A method of measuring the translation quality of ancient Chinese characters by an automatic translator based on .
The method of claim 1, wherein the normalization process of the normalized translation text in the normalized translation text generation step and the normalization process of the normalized evaluation set in the normalized evaluation set generation step are:
A morpheme analysis step of analyzing the morphemes of the input Hangul sentence,
After the morpheme analysis step, a representative word search step of searching for representative words according to morphemes in the representative word database,
A representative word substitution step in which representative words searched in the representative word search step are replaced with each translation;
A method of measuring the translation quality of archaic Chinese characters by an automatic translator based on representative words, which is characterized by consisting of a normalized sentence completion step of completing the normalized sentence through the representative word substitution step.

Images