KR20110062867A - Apparatus and method for constructing terms list of source language-target language - Google Patents

Abstract

PURPOSE: A source language-object language term list constructing apparatus and method thereof are provided to construct an original language-object language term list through mining of a search result through an intermediate language. CONSTITUTION: A document search unit(105) collects a search result including an original language. A data mining unit(107) extracts intermediate language parallel sentence or a word. A morpheme analysis unit(109) analyzes an extracted original language-middle language parallel sentence. A word alignment unit(111) aligns the language parallel sentence. A term generator(113) generates an original language term list.

Description

APPARATUS AND METHOD FOR CONSTRUCTING TERMS LIST OF SOURCE LANGUAGE-TARGET LANGUAGE}

The present invention relates to an apparatus and method for constructing a source language-target language term list, and more particularly, to a source language-target language term list in which a source language term and a target language term corresponding to the target language term are matched. The present invention relates to an apparatus and method for constructing a primitive language-purpose language terminology for mining and constructing a search result using a pivot language.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy [Task management number: 2009-S-034-01, Task name: Korean-Chinese dialogue and corporate document automatic translation technology development].

In recent years, with the rapid development of technology, technical exchanges between languages have deepened. The deepening of technical exchanges between specific languages promotes the demand for specialized translation technology. One example is the need for automatic translation of specialized fields such as patents and papers. However, specialized field translation is difficult to build a terminology list that determines the translation performance, and the frequent appearance of new terminology causes a decrease in translation performance.

The following two methods are largely used to improve the performance of a machine translation system according to the prior art.

First, the source language corpus is collected, new words (including terminology) extracted from the collected source language corpus are extracted from the new words that do not exist in the dictionary including the terminology, and the expert translator asks the band words for the extracted new words. The terminology list was constructed in the same way. Recently, a method of extracting a new word using a parallel corpus composed of a pair of primitive languages and a target language has also been used. These methods are suitable for use in domains where documents are easy to collect, such as news, and premise that source technology should be collected in technical fields. However, specialty documents are difficult to collect because they are technically protected, and specialty documents in languages such as Chinese are more difficult to collect. In addition, even if a translator fluent in bilingualism requires a lot of expertise in selecting a terminology and requesting a band word, it is difficult, and it is difficult to construct a high-quality terminology.

Second, there is a method of estimating a band word of a new word through web mining. The characteristic of this method is that it is based on a pattern of "primary language word (target language word)" form of the web search result through a web search for a known word. This method is attempted in cross-language information retrieval to estimate and extract band words. However, this method needs to know the source language coincidence in advance, and a pattern such as "native language word (object language)" must exist in the web search result to estimate and extract the language.

The present invention has been proposed to solve this problem of the prior art, by mining the search results using the intermediate language to build a list of primitive language-purpose language terms, in particular the automatic field of expertise between non-English language (eg Chinese, Korean) Make it easy to build a list of terminology that is not in the dictionary in the translation system.

In accordance with a first aspect of the present invention, an apparatus for constructing a primitive language-target language terminology includes: a term extractor which extracts an intermediate language term from an intermediate-object language term list; A document retrieval unit for collecting search results including a primitive language among the results, a data mining unit for extracting a primitive language-middle language parallel sentence or word by mining the search result, and a parallel sentence of the extracted primitive language-middle language Or a morpheme analysis unit which performs source language morphological analysis and intermediate language morphological analysis on each word, and the source language-middle language parallel sentences or words on which the morphological analysis is performed are arranged in word units, and the intermediate language in the search result. A word aligning unit for aligning the term and the defensive source language bandword, and the intermediate language sorted Prototype-target language term list is generated by selecting target language-intermediate language pairs corresponding to the selected source language by selecting source language-middle language term according to pre-set selection criteria for language terminology and common source language band language. It may include a term generating unit.

Here, when the search query is included in the intermediate language sentences included in the search results, the data mining unit may extract both a source language sentence and an intermediate language sentence including the search query.

The data mining unit estimates whether the source language sentence and the intermediate language sentence are matched using sentence length or punctuation, and removes the search result that is not matched by filtering.

The data mining unit may estimate and extract the immediately preceding word as a corresponding source language term when the search query word is included in a source language sentence included in the search result.

The term generator may set at least one or more of the frequency of the source language term, whether to include unnecessary investigation of the source language term, and the difference between the word size of the intermediate language term and the original language term as the selection criteria.

In accordance with a second aspect of the present invention, a method for constructing a source language-target language term list includes extracting an intermediate language term from a middle-object language term list, and performing a data search using the extracted intermediate language term as a search query. Collecting a search result including a source language, mining the search result to extract a primitive language-medium language parallel sentence or word, and primitive for each of the extracted primitive language-medium language parallel sentences or words Performing language morphological analysis and intermediate language morphological analysis, and sorting the primitive-intermediate parallel sentences or words on which the morphological analysis has been performed by word units to form intermediate language terms and common primitive band words in the search results. Sorting the target intermediate language and the defensive primitive language; The method may include generating a source language-target language term list corresponding to the selected source language by selecting the source language-intermediate language term according to a predetermined selection criterion.

The extracting of the primitive-intermediate language parallel sentences or words may include extracting the primitive sentence and the intermediate sentence including the search query if the search query is included among the intermediate sentences included in the search results. Can be extracted.

The extracting of the source language-medium language parallel sentence or word may include estimating whether the source language sentence and the intermediate language sentence correspond to each other using sentence length or punctuation and then remove the unmatched search result through filtering. can do.

The extracting of the source language-medium language parallel sentences or words may include extracting the previous word as a corresponding source language term if the search query is included in the source language sentence included in the search result. Can be.

The generating of the primitive language-target language term list may include at least one of a frequency of the primitive language term, whether to include unnecessary investigation of the primitive language term, and a difference in word size between the intermediate language term and the primitive language term. Can be set as selection criteria.

According to an embodiment of the present invention, by constructing a list of source language-target language terms by mining a search result using an intermediate language, a term list for an automatic translation system can be easily constructed, and in particular, a non-English language (eg, Chinese, It is possible to easily build up the terminology list that is not in the dictionary in the automatic translation system.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various different forms, and only the embodiments make the disclosure of the present invention complete, and the general knowledge in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like numbers refer to like elements throughout.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, terms to be described below are terms defined in consideration of functions in the embodiments of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be based on the contents throughout this specification.

Combinations of each block of the accompanying block diagram and each step of the flowchart may be performed by computer program instructions. These computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment such that instructions executed through the processor of the computer or other programmable data processing equipment may not be included in each block or flowchart of the block diagram. It will create means for performing the functions described in each step. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flow chart step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.

Apparatus and method for constructing a source language-target language terminology list according to the present invention can be used to construct a language list for an automatic translation system between non-English speaking languages (eg, Chinese and Korean), as well as language for all languages without any kind of language. Applicable to building a list of terms. In the following description, an embodiment using English as an intermediate language for constructing a Chinese-Korean term list will be described to help understand the description. In other words, English is an intermediate language, an original language is Chinese, and the target language is Korean.

1 is a block diagram of an apparatus for constructing a source language-target language terminology according to an embodiment of the present invention.

As shown therein, the primitive language-target language term list construction device includes an English-English terminology dictionary 101, an English term extraction unit 103, a document search unit 105, a data mining unit 107, and a morphological analysis unit 109. , A word aligning unit 111, a Chinese term generating unit 113, a central term dictionary 115, and the like.

The English term extracting unit 103 extracts an English term from the English-English term list stored in the English-English term dictionary 101 and provides it to the document search unit 105.

The document search unit 105 collects a search result including Chinese from the data search result using the English term extracted by the English term extraction unit 103 as a search query and provides the search result to the data mining unit 107.

The data mining unit 107 mines the search result by the document searching unit 105, extracts the middle-parallel parallel sentences or words, and provides them to the morpheme analysis unit 109. If a search query is included among the English sentences included in the search results, the Chinese sentence and the English sentence containing the search query are extracted together, and the sentence length or punctuation for the Chinese sentence and the English sentence is estimated and not responded. If the search result is not removed through filtering, and the search query is included in the Chinese sentence included in the search result, the previous word is estimated by the corresponding Chinese term and extracted.

The morpheme analysis unit 109 performs Chinese morphological analysis and English morphological analysis on the Chinese-English parallel sentences or words extracted by the data mining unit 107, and provides the result to the word alignment unit 111.

The word sorting unit 111 sorts the English-Chinese parallel sentences or words on which the morphological analysis has been performed in units of words, and arranges the English terms in the search results and the Chinese Traditional Chinese words.

The Chinese term generation unit 113 generates Korean Korean word pairs corresponding to the selected Chinese by selecting Chinese and English terms according to a predetermined selection criterion for the English terms arranged by the word aligning unit 111 and the Chinese Traditional Chinese words. Create a list of harmful terms. Here, the frequency of Chinese terms, whether or not to include unnecessary investigation of Chinese terms, the difference in the word size of English terms and Chinese terms, etc. are used as the selection criteria for Chinese-English terms.

2 is a flowchart illustrating a method of constructing a source language-target language term list according to an embodiment of the present invention.

As described above, the method for constructing a primitive language-target language term list includes extracting an English term from an English-English term list (S201), and selecting a search result including Chinese from a result of data search using the extracted English term as a search query. Collecting (S203, S205), mining the search results to extract the Chinese-Parallel parallel sentences or words, and performing Chinese morphological analysis and English morphological analysis on the extracted Chinese-Parallel parallel sentences or words, respectively. Step S213, sorting the English-language parallel sentences or words which have been subjected to morphological analysis by word units, and sorting the English terms and the simplistic Chinese bandwords in the search results (S215), the sorted English terms and the simplistic Chinese bandwords Korean language corresponding to Chinese selected by selecting Chinese-English terminology according to preset criteria To create a pair yeokeo and a step (S217, S219) for generating a serious term list.

Hereinafter, a process of constructing an important term list by the apparatus for constructing a source language-target language terminology according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

First, the English term extracting unit 103 extracts an English term from the pre-built English-Korean term dictionary 101 and transmits it to the document search unit 105 (S201). Here, the English-Korean term dictionary 101 means that the term list matching the English term and the Korean term, which is the band word, is electronically aggregated.

The document search unit 105 performs an online data search using the English term received from the English term extraction unit 103 as an English query (S203), and collects a search result document including Chinese from the data search results and collects the data. It transfers to the mining part 107 (S205). For example, Chinese academic documents containing English terms are collected when an English query is searched for academic sites using Chinese as the main language (eg, http://scholar.google.com). Here, the use of English as a query is because most of the terminology begins with English, and in academic papers in non-English languages such as Chinese and Korean, titles and summaries are written in English and local language pairs, or terminology. I would like to take advantage of features that often describe "local language". Similarly, most Chinese academic documents include English in their titles and summaries. In addition, technical documents written in Chinese are often written in a pattern in the form of "Chinese (English)" for a particular terminology, so that Chinese documents containing English terms You can search.

The data mining unit 107 mines the search result document collected by the document searching unit 105. If an English query is included in the English sentences included in the contents of the search result document, that is, if the surrounding words are English, the Chinese sentences of the search result document and the English sentences including the English query are extracted together. At this time, whether the Chinese sentence corresponds to the English sentence is estimated using the length of the two sentences, the sentence code and the like. In this case, when a condition in which a Chinese sentence corresponding to an English sentence exists does not satisfy (S207), that is, when there is no Chinese sentence corresponding to the English sentence, the corresponding search result document is removed through filtering (S209).

In addition, if the English query is included in the Chinese sentence included in the content of the search result document, the data mining unit 107 estimates and extracts the immediately preceding word as a corresponding Chinese term. Since Chinese does not use spaces between words, it is difficult to know how far the previous Chinese corresponds. However, since most Chinese words are composed of 2 or 3 letter words, for example, the previous word is extracted by setting the extraction size to 3 letters. In this way, the Chinese-English parallel sentences or words extracted by the data mining unit 107 are provided to the morpheme analysis unit 109 (S211).

The morpheme analyzing unit 109 performs Chinese morphological analysis and English morphological analysis on the Chinese-Chinese parallel sentences or words provided from the data mining unit 107 and provides the result to the word alignment unit 111 (S213). This is because the Chinese sentence is not a word unit segment, in the case of English to extract the morphemes. In addition, it increases the alignment ratio of word alignment to be performed later.

The word alignment unit 111 performs word unit alignment on the stemmed Chinese and English sentences or words. At this time, the alignment of word units is performed by using a tool such as GIZA ++ which is commonly used in the alignment between heterogeneous languages. When the word unit is sorted in this way, the English term in the search result document and the Chinese Traditional Chinese word are sorted at a high frequency and are provided to the Chinese term generator 113 (S215).

The Chinese term generator 113 selects the English term and the Chinese band word arranged by the word aligning unit 111 as the Chinese-English term in the unit of words, and sets the English-English term dictionary 101 to match the English term of the selected Chinese band word. (S217) to finally build a heavy term dictionary (115) (S219). Here, the selection of Chinese terms is mainly performed by checking whether the frequency of Chinese terms is high, whether unnecessary investigations are included in Chinese terms, and / or whether the word size is too different from the corresponding English terms. Select.

1 is a block diagram of an apparatus for constructing a source language-target language terminology according to an embodiment of the present invention;

2 is a block diagram of a method for constructing a primitive language-purpose language terminology list according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

101: English-English term dictionary 103: English term extraction unit

105: document search unit 107: data mining unit

109: stemming unit 111: word alignment unit

113: Chinese terminology generation unit 115: Chinese terminology dictionary

Claims (10)

A term extraction unit for extracting an intermediate language term from the intermediate-object language term list; A document search unit which collects a search result including a source language among data search results using the extracted intermediate language term as a search query; A data mining unit which extracts source language-medium language parallel sentences or words by mining the search results; A morphological analysis unit for performing source language morphological analysis and intermediate language morphological analysis on the extracted parallel sentences or words of the source language-middle language, respectively; A word alignment unit for arranging the primitive-middle language parallel sentences or words on which the morphological analysis has been performed, on a word-by-word basis, to align the intermediate language term and the unfamiliar primitive language band word in the search result; Proto-language-target language is generated by selecting a target language-intermediate language pair corresponding to the selected source language by selecting the source language-intermediate language term according to the selected selection criteria for the intermediate language term and the primitive source language band. Including a term generator for generating a term list Source language-target language term list building device. The method of claim 1, The data mining unit extracts a source language sentence and an intermediate language sentence including the search query if the search query is included among the intermediate language sentences included in the search result. Source language-target language term list building device. The method of claim 2, The data mining unit estimates whether the source language sentence and the intermediate language sentence are matched using sentence length or punctuation, and removes the unmatched search result through filtering. Source language-target language term list building device. The method of claim 2, The data mining unit estimates and extracts the immediately preceding word as a corresponding source language term when the search query word is included in a source language sentence included in the search result. Source language-target language term list building device. The method of claim 4, wherein The term generator is configured to set at least one or more of the frequency of the source language term, whether to include unnecessary investigation of the source language term, and the difference between the word size of the intermediate language term and the source language term as the selection criteria. Source language-target language term list building device. Extracting the intermediate language term from the intermediate-object language term list; Collecting search results including a source language from data search results using the extracted intermediate language terms as a search query; Mining the search results to extract source language-medium language parallel sentences or words; Performing source language morphological analysis and intermediate language morphological analysis on the extracted parallel sentences or words of the source language-middle language, respectively; Arranging the source language-medium language parallel sentences or words on which the morphological analysis has been performed in word units to align intermediate language terms and common source language band words in the search results; Proto-language terminology is generated by generating a pair of target language bands corresponding to the selected primitive language by selecting the primitive language-middle language term according to a predetermined selection criterion for the intermediate and linguistic primitive band languages. Generating a list How to build a list of source language-target language terms. The method of claim 6, The extracting of the source-middle parallel sentences or words may include extracting a source language sentence and an intermediate language sentence including the search query when the search query is included in the intermediate language sentences included in the search results. How to build a list of source language-target language terms. The method of claim 7, wherein The extracting of the source language-medium language parallel sentence or word may include estimating whether the source language sentence and the intermediate language sentence correspond to each other using sentence length or punctuation and then remove the unmatched search result through filtering. doing How to build a list of source language-target language terms. The method of claim 7, wherein The extracting of the source language-medium language parallel sentences or words may include extracting by estimating the immediately preceding word as a corresponding source language term if the search query word is included in the source language sentence included in the search result. How to build a list of source language-target language terms. The method of claim 9, The generating of the primitive language-target language term list may include at least one of a frequency of the primitive language term, whether to include unnecessary investigation of the primitive language term, and a difference in word size between the intermediate language term and the primitive language term. Set by selection criteria How to build a list of source language-target language terms.

Images