KR100617319B1 - Apparatus for selecting target word for noun/verb using verb patterns and sense vectors for English-Korean machine translation and method thereof - Google Patents

Abstract

Verb / noun band word selection device and method using verb phrase pattern and semantic vector for English-Korean automatic translation according to the present invention are based on verb phrase pattern based on English-Korean parallel corpus, English monolingual corpus and wordnet Building a database; Constructing a semantic vector and Korean local context information after performing lexical alignment on each pair of sentences in a parallel corpus composed of a pair of original and band sentences; Dividing an input sentence into short sentence units and searching for a verb phrase pattern to be applied to the short sentence by referring to the verb phrase pattern database; Determining a band word of a verb and a noun semantic code of each argument based on the found verb phrase pattern if the search is successful; If the search fails and the verb phrase pattern is not applied, selecting a basic verb band word by applying a predetermined default verb phrase pattern; And determining a semantic code using an English air vocabulary if the search fails and the argument of the verb is a noun phrase. The band word for the verb and the head noun is selected by the verb phrase pattern. Air information also allows the selection of bandwords for other nouns other than head nouns.

Machine translation, automatic translation, band choice, semantic vector, verb phrase pattern

Description

Apparatus for selecting target word for noun / verb using verb patterns and sense vectors for English-Korean machine translation and method

1A is a flowchart for constructing a verb phrase pattern for selecting a English-Korean verb band word.

1B is a diagram illustrating an example of a format of an English-Korean verb phrase pattern applied to the present invention.

2 is a flowchart illustrating a process of configuring a semantic vector database and a local context information database according to the present invention.

3 is a block diagram showing the configuration of a verb / noun band word selection device using a verb phrase pattern and a semantic vector according to the present invention.

4 is a flowchart illustrating a process of a verb / noun band word selection method using a verb phrase pattern and a semantic vector according to the present invention.

* Explanation of symbols for the main parts of the drawings

301: short sentence division unit 302: verb phrase pattern application unit

304: verb band word determination unit 305: noun band word selection unit

307: test vector component 308: similarity calculator

309: noun semantic code determiner 311: default verb phrase pattern applying unit

303: verb phrase pattern database 310: semantic vector database

306: Korean Local Context Database

The present invention provides an apparatus and method for selecting a noun and a band word of a verb using a verb phrase pattern and a semantic vector in an English-Korean automatic translation system, and a computer-readable recording medium recording a program capable of executing the method on a computer. It is about.

In the case of an automatic translation system, the technique of translating a band word without losing the unique meaning of each vocabulary in the original text is a very important factor in obtaining a natural translation quality. To date, bandword selection techniques have been provided, which mainly rely on simple air information, but the performance is insufficient to obtain a practical automatic translation result.

At the same time, some techniques have been proposed to solve the semantic ambiguity of the vocabulary of the primitive language, but when selecting the band word in the automatic translation, only the semantic ambiguity of the original vocabulary cannot be solved to obtain the natural band word, and the aspect of the target language must be considered. do.

Therefore, in order to solve the above problems, first, the semantic ambiguity of the original vocabulary must be solved, and among the band candidates having the meaning determined in this step, a method of selecting a band word most suitable for the Korean context to be generated is required.

The technical problem to be achieved by the present invention is to solve the above problems, it is a core part of speech in the English-Korean automatic translation and when selecting the nouns and verb band words that greatly influence the performance of the entire translation fits the context of the sentence. An apparatus, a method, and a method for selecting the most natural band word are provided.

In order to achieve the above technical problem, a verb / noun band word selection method using a verb phrase pattern and a semantic vector for the English-Korean automatic translation according to the present invention uses a semantic code based on an English-Korean parallel corpus, an English monolingual corpus, and WordNet. Constructing a verb phrase pattern database as a basis; Constructing a semantic vector and Korean local context information after performing lexical alignment on each pair of sentences in a parallel corpus composed of a pair of original and band sentences; Dividing an input sentence into short sentence units and searching for a verb phrase pattern to be applied to the short sentence by referring to the verb phrase pattern database; Determining a band word of a verb and a noun semantic code of each argument based on the found verb phrase pattern if the search is successful; If the search fails and the verb phrase pattern is not applied, selecting a basic verb band word by applying a predetermined default verb phrase pattern; And if the search fails and the argument of the verb is a noun phrase, determining a semantic code using an English air vocabulary.

 In order to achieve the above technical problem, a verb / noun band word selection device using a verb phrase pattern and a semantic vector for English-Korean automatic translation according to the present invention comprises: a short sentence division unit for receiving a sentence and dividing it into a short sentence unit; A verb phrase pattern application unit for searching a verb phrase pattern applicable to the short sentence by referring to a verb phrase pattern database; A verb band word determining unit configured to determine verb band words and meaning codes of nouns by receiving verb verb patterns searched by the verb phrase pattern applying unit if the search is successful; And a band word selector that probabilistically selects and outputs a band word by referring to a semantic vector database and a Korean local context database if the search in the verb phrase pattern application unit fails.

 In order to achieve the above technical problem, a computer-readable recording medium recording a program capable of executing a verb / noun band word selection method using a verb phrase pattern and a semantic vector for the English-Korean automatic translation according to the present invention is in parallel with English. Constructing a verb phrase pattern database based on semantic codes based on corpus, English monolingual corpus, and WordNet; Constructing a semantic vector and Korean local context information after performing lexical alignment on each pair of sentences in a parallel corpus composed of a pair of original and band sentences; Dividing an input sentence into short sentence units and searching for a verb phrase pattern to be applied to the short sentence by referring to the verb phrase pattern database; Determining a band word of a verb and a noun semantic code of each argument based on the found verb phrase pattern if the search is successful; If the search fails and the verb phrase pattern is not applied, selecting a basic verb band word by applying a predetermined default verb phrase pattern; And determining a semantic code using an English air vocabulary if the search fails and the argument of the verb is a noun phrase. The program may be executed by a computer.

Prior to describing the preferred embodiment of the present invention, the concept of terms used in the detailed description and claims are summarized as follows.

The English-Korean parallel corpus is composed of an English sentence and a Korean band sentence, and can be said to be the main extraction target of the bandword selection knowledge used in the present invention.

The English monolingham corpus is a monolingual corpus consisting only of English sentences and is used to solve the data shortage problem of relatively small parallel corpus.

The verb phrase pattern is composed of the original English pattern and the Korean band pattern, and it can be called a pattern that includes grammatical elements that the verb takes as an argument centering on the English verb. Is used.

Noun vocabulary alignment can be referred to as a task of mapping the Korean noun vocabulary to the English noun vocabulary for each pair of English sentences and Korean sentences constituting the English-Korean parallel corpus.

A semantic vector is composed of N-dimensions, the elements of which represent the weighted values of each English noun and the lexical vocabulary, and this information is extracted from the English-Korean parallel corpus. The dimension of the semantic vector is defined as the number of vocabulary having a high relationship with the English noun vocabulary based on mutual information (MI).

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before describing each embodiment of the Korean nouns of English nouns and verbs according to the present invention, descriptions of verb phrase patterns and semantic codes are given priority.

The verb phrase pattern is composed of the original part and the band part as shown in Example 1 below.

[Example 1] Example of verb phrase pattern in English verb abandon

S = (person # 1) abandon O = (idea # 1) => S! Gives up O!

Example 1 above shows one of the verb phrase patterns of the English verb 'abandon'. In the above verb phrase pattern, the original part is "S = (person # 1) abandon O = (idea # 1)", and the band part is "S! Gives up O!". In the text, symbols such as S and O represent arguments, S is given, and O is the object. Person # 1 and idea # 1 also represent possible semantic codes in place of arguments. To explain in advance, in order to apply the verb phrase pattern corresponding to Example 1 to a sentence using 'abandon' as the main verb, the semantic code corresponding to the subject and object of the sentence matches the semantic code described in the verb phrase pattern. Should be. When the verb phrase pattern is matched, the verb band word is determined through the band portion of the verb phrase pattern, and in the case of a matching noun in the argument, the corresponding semantic code is determined.

The following describes semantic features for each entry of the English-Korean dictionary used in the present invention. In the case of the English-Korean dictionary, the semantic code feature is named 'SEM', which is assigned one of 1,163 semantic codes that follow the semantics of WordNet1.71. Then, a method of classifying the band word according to a semantic code is disclosed.

In general, English vocabulary has many polysemous words that can be translated into two or more Korean words. Table 1 shows examples of representative English nouns.

Vocabulary Meaning Code Band word goal / NOUN goal # 1 Goals, results, objectives equipment # 1 goal accomplishment # 1 score extremity # 4 Finish line, destination, destination

Table 1 shows examples of bandword classification according to semantic codes for English vocabulary goal / NOUN. In Table 1, the English vocabulary goal / NOUN has four semantic codes: goal # 1, equipment # 1, accomplishment # 1, and extremity # 4, and Korean Korean words corresponding to each semantic code are classified as shown in Table 1, respectively. .

 FIG. 1A is a flowchart for constructing a verb phrase pattern for selecting a Korean-English verb band word, and FIG. 1B is a diagram illustrating an example of a format of an English-Korean verb phrase pattern applied to the present invention. This shows a verb phrase pattern that is knowledge for selecting English-Korean verb band words to which the present invention is applied. Referring to FIG. 1A, the verb phrase pattern process first extracts argument information taken by a verb based on the verb (S101). Next, a semantic code is assigned to the extracted argument information (S102), and a Korean band pattern is constructed for the English verb phrase pattern (S103). The format of the English-Korean verb phrase pattern is also described in FIG. 1B.

FIG. 2 is a diagram illustrating an embodiment of a method of constructing a semantic vector and Korean local context information, which are knowledge for band word selection, in an apparatus for selecting a Korean-English band word using a semantic vector to which the present invention is applied. For convenience of description, the description of the configuration of the device and the flow description of the method will be described together.

Referring to FIG. 2, in the present invention, a morpheme analysis / tagging operation is performed using an English morpheme analyzer, an English tagger, a Korean morpheme analyzer, and a Korean tagger for each English sentence and a Korean sentence of a parallel corpus (not shown) ( S201).

The nouns are sorted using information of the English-Korean electronic dictionary for each sentence of the tagged parallel corpus (S203).

The present invention refers to the English-Korean electronic dictionary, and it is possible to know what semantic codes each band word has, and by using this information, a semantic vector is constructed (S204) and Korean local context information is constructed (S205). Information is stored in the semantic vector database (hereinafter referred to as "DB") 310 and the Korean local context information DB 306.

The semantic vector is composed of N-dimensional vectors whose components represent the weights of each English noun and the lexical vocabulary, and this information is extracted from the parallel corpus. At this time, the parts of speech permitted as the air information are limited to adjectives and nouns. The reason is that the vocabulary of functional fish, such as prepositions, has no discernment in determining meaning. The dimension N of the semantic vector is defined as the number of vocabulary having a high relationship with the English noun vocabulary based on mutual information (MI).

Equation 1 below represents mutual information, Equation 2 indicates a semantic vector SV, and Equation 3 indicates its components.

 Where si is arbitrary

Equation 1 represents a formula for mutual information MI, and Equation 2 represents a semantic vector and its components. In Equation 2, a weight value for each component is shown in Equation 3. That is, each component of the semantic vector represents a conditional probability value of the meaning for the air vocabulary, and the value has a value between 0 and 1, and the closer to 1, the co-occurring word is the English noun vocabulary. This is a very powerful clue for determining a particular meaning. These semantic vectors are made up of different semantic codes for each English noun vocabulary. For example, for 'race / NOUN' shown in Table 1, two semantic vectors are created.

The Korean local context information is used to select an optimal band word when there are two or more Korean band words having the same semantic code as shown in Table 1. As shown in Figs. 1A to 1B, Korean local context information is constructed using only Korean parts of a parallel corpus, and represents frequency information of mutual airing only for adjectives and nouns.

 Figure 3 is a block diagram showing the configuration of a verb / noun band word selection apparatus using a verb phrase pattern and a semantic vector according to the present invention, Figure 4 is a verb / noun band word selection method using a verb phrase pattern and a semantic vector according to the present invention Is a flow chart showing the process. First, as described above, the verb phrase pattern DB 303, the semantic vector DB 310, and the Korean local context DB 306 are constructed (S401). Now, the input sentence is first divided into short sentences by the short sentence divider 301. The verb phrase pattern applying unit 302 receives the divided short sentence and searches the verb phrase pattern DB 303 centered on the verb of each short sentence and searches for the existence of the verb phrase pattern covering the short sentence and attempts to apply the verb phrase pattern ( S402).

In other words, if there is a matching verb phrase pattern (S403), the verb band word determination unit 304 determines a verb band word based on the matched verb phrase pattern, and determines a semantic code of a noun located in the argument of the verb. (S404). After the semantic code of the noun is determined, it is determined whether the noun has two or more Korean band words belonging to the determined semantic code (S405), and if there are two or more Korean band words, the noun band word selection unit 305 determines the Korean local context. With reference to the DB 306, the Korean band word most suitable for the context of the input sentence is selected (S411).

As another case, that is, when a matching verb phrase pattern is not searched, it is determined below whether the argument of the verb is a noun phrase, a verb or a noun (S406), and goes through a different process.

First, the arguments of verbs are made up of noun phrases to determine the semantic codes for non-head nouns other than head nouns, and the semantic codes are determined using English air vocabulary. That is, the test vector constructing unit 307 of the present invention creates a test vector having the same dimension as the semantic vector of the corresponding vocabulary for the vocabulary having the bandword selection ambiguity in the conversion step from the input sentence (S408).

The similarity calculation vector between the vectors 308 calculates the similarity between the test vector obtained from the test vector constructing unit 307 and the semantic vector already constructed (S409), and the noun semantic code determining unit 309 calculates the noun semantic code. Determine (S410). After the semantic code is determined, as described above, if the noun has two or more band words belonging to the determined semantic code, the noun band word selection unit 305 uses the Korean local context DB 306 to determine the final result. A Korean band word is selected (S411).

Finally, when the verb phrase pattern is not applied, in case of the verb, the default verb phrase pattern applying unit 311 selects a basic verb band word using the default verb phrase pattern (S407), and in case of noun, compares with a semantic vector using air information. The semantic code is determined by the similarity calculation (S409) (S410), and the final band word is selected by the noun band word selection unit 305.

A method of selecting a band word of the corresponding English vocabulary using the semantic vector and the Korean local context information described above will be described in detail as follows.

Referring to the drawing, when an input statement is input to the test vector constructing unit 307, the test vector constructing unit 307 has a test vector having the same dimension as the semantic vector of the corresponding vocabulary in the converting step. Make Each component of the test vector has a value of 0 or 1, and 1 if the air vocabulary for each English noun vocabulary included in the input sentence is a component of the semantic vector of the corresponding English noun vocabulary. In case it has 0. For example, suppose that the semantic vector of 'bank / NOUN' is (w (rain), w (commercial), w (money)), and the input sentence is "Rain broke the bank" (1, 0, 0 A test vector is generated.

The similarity between the test vector obtained from the input sentence and the previously constructed semantic vector uses a cosine measure, and the vector similarity calculator 308 calculates similarity between the two vectors according to Equation 4 below. Calculate

As described above, the semantic code is determined by the noun semantic code determining unit 304 with respect to the English noun vocabulary having the bandword selection ambiguity as described above (S410), and finally, the Korean bandword having the semantic code of the determined vocabulary is 2 If there are more than one, the noun band word selection unit 305 uses the information stored in the Korean local context information DB 306 to probabilistically calculate a band word that is semantically closest to an adjacent noun or adjective to determine an optimal band word. .

Example 2 shows various cases in which the English noun 'change / NOUN' is translated into Korean.

[Example 2]

interest rate changes (interest rate)

changes in your cells (changes in the cell)

requests for services changes (Service Change Request)

As can be seen in Example 2, the band words 'change', 'change' and 'change' of the English noun 'change / NOUN' cannot be determined simply by determining the meaning of the English vocabulary 'change / NOUN'. This is because the band words 'change', 'change', and 'change' of 'change / NOUN' have the same semantic code, and such subtle Korean bandword selection can only select the most natural bandword by considering Korean local context information.

As mentioned above, resolving subtle differences in bandword selection relies on Korean local context information built from the Korean part of the parallel corpus. The Korean local context information is composed of air frequencies consisting of adjectives and nouns, and the optimal Korean band words are selected by obtaining probability values for sequences of Korean vocabularies passed to the generation part of the English-Korean automatic translation system.

The verb / noun band word selection method using a verb phrase pattern and a semantic vector for the English-Korean automatic translation according to the present invention can also be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, which are also implemented in the form of a carrier wave (for example, transmission over the Internet). It also includes. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible in the technical field to which the present invention pertains without departing from the technical spirit of the present invention. It will be clear to those of ordinary knowledge.

As described above, according to the present invention, a verb / noun band word selection device using a verb phrase pattern and a semantic vector and a method for constructing an English-Korean automatic translation according to the present invention constitute a sentence composed of a verb phrase pattern constructed from an English-Korean parallel corpus and an English monolinge corpus. The verbs and verbs of verbs and nouns are obtained by applying to each short sentence, and the parallel corpus of nouns whose meaning is not determined by the verb phrase pattern is not applied because the verb phrase pattern is not applied or the head vocabulary of the noun phrase is not applied when the verb phrase pattern is applied. The meaning of the English vocabulary is determined using the semantic vector obtained by the vocabulary sorting process. After the meaning of the noun is determined in this way, among the band words of the corresponding English vocabulary, it is possible to select a Korean band word most natural to the context using the existing Korean local context information. It is also possible to select bandwords for verbs and head nouns by verb phrase patterns, and bandwords for other nouns other than head nouns by air information.

Claims (15)

(a) constructing a verb phrase pattern database based on semantic codes based on English-Korean parallel corpus, English monolingual corpus and WordNet; (b) constructing a semantic vector and Korean local context information after performing lexical alignment on each pair of sentences in a parallel corpus composed of a pair of original and band sentences; (c) dividing an input sentence into short sentence units and searching for a verb phrase pattern to be applied to the short sentence by referring to the verb phrase pattern database; (d) if the search is successful, determining a band word of a verb and a noun semantic code of each argument based on the found verb phrase pattern; (e) selecting a basic verb band word by applying a predetermined default verb phrase pattern when the search fails and the verb phrase pattern is not applied; And (f) determining a semantic code using an English air vocabulary if the search fails and the term of the verb is a noun phrase; and includes a verb phrase pattern, a semantic vector, and Korean local context information in an English-Korean automatic translation. How to select English verbs / noun bands used. The method of claim 1, wherein step (a) (a1) extracting argument information taken by the verb mainly from the verb; (a2) tagging semantic information on the extracted dispute information based on WordNet; (a3) constructing Korean band information for sub-category information of the verbs; an English verb / noun band word selection method using a verb phrase pattern, a semantic vector, and Korean local context information in an English-Korean automatic translation. The method of claim 1, wherein step (b) (b1) analyzing and tagging morphemes of English and Korean sentences from the parallel corpus; And (b2) constructing the semantic vector after arranging the lexical levels of the tagged pairs with reference to a predetermined English-Korean electronic dictionary; a verb phrase pattern, a semantic vector, and A method of selecting English verb / noun band words using Korean local contextual information. The method of claim 3, wherein the semantic vector An English verb / noun band word selection method using verb phrase patterns, semantic vectors, and Korean local context information in English-Korean automatic translation, characterized by representing the weighted values of English nouns, each English noun, and the lexical vocabulary. The method of claim 1, wherein step (d) Selecting an optimal band word by referring to the Korean local context information database when there are two or more Korean band words; and the English verb using a verb phrase pattern, a semantic vector, and Korean local context information in the English-Korean automatic translation. / Noun bandword selection method. The method of claim 1, wherein step (f) (f1) constructing a predetermined test vector for each vocabulary; (f2) determining a semantic code by calculating a similarity between the test vector and the semantic vector; And (f3) if the noun having a semantic code has two or more band words, selecting an optimal band word with reference to the Korean local context database; and includes a verb phrase pattern, a semantic vector, and a Korean word in an automatic Korean-English translation. How to select English verb / noun band words using local contextual information. The method of claim 6, wherein the test vector is The verb phrase in the English-Korean automatic translation, characterized in that it has a value of 1 if the air vocabulary for each English noun included in the input sentence is a component of the semantic vector of the corresponding English noun vocabulary. A method of selecting English verb / noun band words using patterns, semantic vectors, and Korean local contextual information. The method of claim 6, wherein step (f2) A method of selecting an English verb / noun band word using a verb phrase pattern, a semantic vector, and Korean local context information in an English-Korean automatic translation, wherein the similarity between the test vector and the semantic vector is calculated using a cosine measure. The method according to any one of claims 5 and 6, The step of selecting an optimal band word by referring to the Korean local context information may include selecting a band word having a probabilistic maximum value of a band word semantically closest to an adjacent noun or adjective by referring to the Korean local context information database. A method of selecting English verb / noun verbs using verb phrase patterns, semantic vectors, and Korean local contextual information. A short sentence division unit that receives a sentence and divides the sentence into short sentence units; A verb phrase pattern application unit for searching a verb phrase pattern applicable to the short sentence by referring to a verb phrase pattern database; A verb band word determining unit configured to determine verb band words and meaning codes of nouns by receiving verb verb patterns searched by the verb phrase pattern applying unit if the search is successful; A verbal word selector which probabilistically selects and outputs a bandword by referring to a semantic vector database and a Korean local context database if the search is unsuccessful in the verb phrase pattern application part; Verb / Noun Band Word Selection Device Using Word, Semantic Vector, and Korean Local Context Information. The method of claim 10, wherein the band word selector A test vector constructing unit generating a predetermined test vector for each vocabulary; A similarity calculator which determines a similarity between the test vector and the semantic vector by referring to the semantic vector database; A noun determination unit for determining and outputting a semantic code of a noun according to the similarity; And A noun band word selection unit for outputting an optimum band word with reference to the Korean local context information database for a noun having two or more semantic codes determined by the uncode determining unit of the noun; Verb / Noun Band Word Selection Device Using Word, Semantic Vector, and Korean Local Context Information. The method of claim 11, wherein the test vector configuration unit If the air vocabulary for each English noun included in the input sentence is a component of the semantic vector of the corresponding English noun vocabulary, the test vector is generated to have a value of 1, otherwise it has a value of 0. English verb / noun band word selection device using verb phrase pattern, semantic vector, and Korean local context information in English-Korean automatic translation. The method of claim 11, wherein the semantic vector constructed in the semantic vector database is An English verb / noun band word selection device using verb phrase patterns, semantic vectors, and Korean local context information in an English-Korean automatic translation characterized by representing weighted values of English nouns and respective English nouns and lexical words. The method of claim 11, wherein the noun band word selection unit English verb / noun bandword selection using verb phrase patterns, semantic vectors, and Korean local contextual information in English-Korean automatic translation characterized by probabilistic calculation of a bandword semantically closest to an adjacent noun or adjective. Device. (a) constructing a verb phrase pattern database based on semantic codes based on English-Korean parallel corpus, English monolingual corpus and WordNet; (b) constructing a semantic vector and Korean local context information after performing lexical alignment on each pair of sentences in a parallel corpus composed of a pair of original and band sentences; (c) dividing an input sentence into short sentence units and searching for a verb phrase pattern to be applied to the short sentence by referring to the verb phrase pattern database; (d) if the search is successful, determining a band word of a verb and a noun semantic code of each argument based on the found verb phrase pattern; (e) selecting a basic verb band word by applying a predetermined default verb phrase pattern when the search fails and the verb phrase pattern is not applied; And (f) determining a semantic code using an English air vocabulary if the search fails and the term of the verb is a noun phrase; and includes a verb phrase pattern, a semantic vector, and Korean local context information in an English-Korean automatic translation. A computer-readable recording medium that records a program that can be run on a computer that uses the English verb / noun band method selection method.

Images